Data Engineer

Overview

We are seeking experienced Data Engineers for our client involved in some of the largest AI, data, and cloud transformation initiatives within the UAE's banking sector. This role is a permanent, onsite position located in Dubai, focusing on leveraging Databricks and cloud technologies to develop scalable and high-performing data solutions.

Key Responsibilities

• Design, develop, and optimize scalable data engineering solutions on Databricks platforms.
• Create and maintain high-performance Apache Spark jobs utilizing PySpark and Spark SQL.
• Develop and sustain robust ETL/ELT pipelines capable of processing extensive structured and semi-structured datasets.
• Ensure reliability, scalability, and quality of data pipelines for smooth operations.
• Produce analytics-ready datasets through comprehensive SQL data modeling techniques.
• Provide timely and accurate data to support Business Intelligence and reporting tools.
• Collaborate closely with analytics teams to gather and interpret business requirements.
• Prepare and curate datasets suitable for AI/ML workloads, supporting feature engineering and model training processes.
• Develop batch and real-time scoring pipelines, working alongside Data Scientists to operationalize AI models.
• Enable end-to-end data workflows facilitating AI-driven projects.
• Engage with Data Science, Analytics, and Platform teams ensuring adherence to data governance, security, and compliance policies.
• Document data pipelines, data models, and operational procedures meticulously.
• Participate in client-facing roles, leveraging banking industry experience to effectively communicate and implement solutions.

Requirements

• Proven expertise with Databricks and Apache Spark development, including PySpark and Spark SQL.
• Hands-on experience designing and implementing data pipelines using Python on Databricks.
• Strong background in handling and processing large-scale structured and semi-structured datasets.
• Solid SQL skills for data modeling and analytics dataset preparation.
• Experience preparing data for AI/ML workflows and working closely with Data Science teams.
• Prior experience in the banking industry with client-facing exposure is essential.
• Knowledge of data governance, security, and compliance standards relevant to banking data projects.

Additional Information

This is a permanent, full-time onsite position based in Dubai. Only candidates with the relevant skills will be contacted due to the volume of applicants.