Data Engineer

Job Overview

We are working with a global investment firm based in Abu Dhabi that is seeking an experienced and proactive Data Engineer. This role involves designing, creating, and managing data pipelines and models that support analytics and applications used by investment teams as well as middle and back office departments.

Key Responsibilities

• Engage actively in requirement gathering and participate in sprint planning.
• Develop and implement technical solutions including end-to-end ETL pipelines.
• Create robust data pipelines utilizing PySpark.
• Enhance and optimize existing ETL processes for better performance.
• Develop comprehensive unit and integration testing for pipelines.
• Provide support to QA teams during the acceptance testing phases.

Required Qualifications and Experience

• Minimum of five years of experience in IT or data roles within a financial services context.
• Bachelor's degree in IT or a closely related discipline is mandatory.

Essential Skills

• Strong analytical skills and exceptional problem-solving capabilities.
• Programming expertise in PySpark for distributed data processing and Python for ETL development.
• Advanced proficiency in SQL, skilled at writing and refining complex queries, with experience in databases like PostgreSQL, MySQL, Oracle, or Snowflake.
• Knowledge of data warehousing principles and platforms, particularly with DataBricks.
• A solid grasp on ETL methodologies and tools.
• Experience in data modeling techniques including dimensional modeling and schema design.
• Ability to manage codebases using version control systems such as Git.
• Familiarity with data pipeline monitoring using tools such as Prometheus and Grafana or custom monitoring utilities.

Additional Information

This is a full-time, permanent position located onsite in Abu Dhabi, United Arab Emirates. Remote work is not available for this role. Candidates applying from outside the region will be offered relocation assistance packages.