Senior Data Engineer

About the Role

We are seeking an experienced Senior Data Engineer to join our onsite team in Abu Dhabi, UAE. This role centers around migrating a large scale data platform using cutting-edge data technologies and creating production-grade pipelines within a fast paced delivery setting. You will be embedded in an engineering team dedicated to producing impactful solutions that demand solid hands-on expertise.

Responsibilities

• Develop, enhance, and deploy data pipelines on Databricks utilizing PySpark and SQL.
• Rebuild and transfer transformation workflows from legacy systems to Databricks.
• Plan and execute data validation and reconciliation to ensure data integrity across platforms.
• Improve pipeline efficiency by applying partitioning, Delta Lake optimization, cluster tuning, and efficient ingestion methods.
• Participate in designing medallion-layer architecture and establish frameworks that ensure data quality.
• Conduct code reviews and provide mentorship to junior engineers to uphold best engineering practices.
• Work within Git-based continuous integration/continuous delivery environments and adhere to Agile methodologies.

Requirements

• A minimum of 7 years experience in data engineering with recent practical involvement.
• Expertise with the Databricks environment, PySpark, SQL programming, and Delta Lake technology.
• Comprehensive understanding of data ingestion techniques, performance tuning for Spark, and partitioning schemes.
• Strong ability to write, analyze, and validate SQL queries.
• Prior experience undertaking data migration or platform reengineering projects.
• Fluent professional English communication skills.
• Bachelor’s degree in Computer Science, IT, or an associated discipline.

Desirable Qualifications

• Certification as a Databricks Certified Data Engineer Professional.
• Experience with Palantir Foundry platform.
• Background knowledge in financial services or asset management sectors.
• Familiarity with Unity Catalog governance model.
• Exposure to comprehensive data quality and reconciliation frameworks for large-scale systems.

Benefits and Perks

• End of service gratuity benefits.
• Private medical and life insurance coverage.
• Employee support and assistance programs.
• Wellness initiatives to promote employee health.
• Annual airfare allowance for expatriate employees.
• Regular performance appraisals coupled with salary reassessments.
• Comprehensive global travel insurance for medical emergencies and accidents.
• Referral reward programs.
• Opportunities for professional growth including internal training sessions, coaching, certification support, and educational courses.