Data Engineer

About the Role

We are seeking an experienced Data Engineer to develop, configure, and fine-tune data ingestion, transformation, and consumption workflows within a modern on-premises lakehouse environment based in Saudi Arabia. The role demands expertise in Spark-based development, enterprise data lake/lakehouse solutions, and Informatica integration technologies.

Key Responsibilities

• Architect and implement scalable pipelines for data ingestion and transformation.
• Build, enhance, and manage Spark-centric ETL/ELT processes.
• Create and maintain data models and datasets leveraging Apache Iceberg tables.
• Design and sustain data integration workflows utilizing Informatica tools.
• Facilitate analytics and data consumption via platforms such as Denodo, Starburst, and Dremio.
• Ensure pipelines are secure, governed, and scalable, complying with enterprise-level standards.
• Handle both batch and near real-time data processing tasks.
• Incorporate data quality, metadata governance, and validation mechanisms.
• Work closely with Platform Engineers on deployment, integration, and operational activities.
• Participate in migration, testing, and deployment of production systems.
• Diagnose and improve pipeline performance, reliability, and scalability.

Required Qualifications and Skills

• Proficiency in Apache Spark (either PySpark or Scala implementations).
• Experience with Apache Iceberg for dataset management.
• Hands-on expertise with Informatica Data Integration tools.
• Knowledge of MinIO object storage systems.
• Familiarity with Denodo for data virtualization and access.
• Strong grasp of SQL and data modeling techniques.
• Understanding of data warehousing principles.
• Competence in ETL/ELT workflow development.
• Experience using Git for version control.

Preferred Skills

• Experience with Starburst or Dremio data query engines.
• Understanding of data governance and metadata management practices.
• Familiarity with data quality frameworks.
• Knowledge of continuous integration and continuous deployment (CI/CD) methodologies for data pipelines.
• Programming skills in Python.
• Experience with Kafka or other streaming technologies.
• Awareness of lakehouse architectural patterns.
• Track record with enterprise-scale data lake and lakehouse projects.
• Skills in performance tuning and scaling of extensive data workloads.