Senior Data Engineer

Overview

We are collaborating with a rapidly expanding technology company to hire a Senior Data Engineer based in Riyadh, Saudi Arabia. This role involves hands-on responsibilities encompassing the design, construction, and maintenance of a comprehensive data platform.

Key Responsibilities

• Architect and manage scalable batch and streaming data pipelines to handle large volumes of data efficiently.
• Develop and refine data architecture covering ingestion, processing, storage, and transformation layers.
• Create dependable and reusable datasets and data models to support various downstream applications and teams.
• Establish data quality processes including validation, monitoring, lineage tracking, and governance protocols.
• Ensure reliability in the data pipelines by implementing idempotency, deduplication, schema evolution handling, recovery mechanisms, and maintaining historical accuracy.
• Set up appropriate access controls to safeguard sensitive information.
• Collaborate with engineering and infrastructure teams to manage and operate production-grade data services.
• Optimize system performance, scalability, and reduce infrastructure costs.
• Engage in developing technical standards, preparing documentation, and participating in code reviews.

Required Qualifications and Experience

• Minimum of 8 years in professional data engineering roles.
• Proficient in Python and SQL, including expertise with PySpark or similar distributed processing frameworks.
• Experienced in AWS data services, particularly with S3, Glue, Glue Data Catalog, Athena, and Redshift.
• Skilled in designing and managing data lake or lakehouse architectures with layered data transformations.
• Hands-on experience with ETL/ELT and data integration tools such as Airbyte, Fivetran, or Meltano.
• Familiarity with streaming and event-driven technologies like Kinesis, MSK, SQS, SNS, DMS, or Debezium.
• Competent in using dbt or similar tools for data transformation and modelling.
• Knowledgeable regarding data storage file formats such as Parquet and modern table formats including Iceberg, Delta Lake, or Hudi.
• Experience with workflow orchestration platforms like MWAA, Step Functions, Dagster, or Prefect.
• Strong grasp of data quality assurance, governance standards, security measures, and lineage tracking.

Additional Expertise

Experience in developing data infrastructure supporting AI or advanced data analytics initiatives is advantageous.