Senior Data Engineer

Job Overview

The Senior Data Engineer is responsible for designing, constructing, and maintaining scalable data pipelines and platforms utilizing Microsoft Fabric and Azure SQL environments. This role supports analytics, artificial intelligence initiatives, and business decision-making through a robust medallion architecture spanning Bronze to Gold layers, ensuring well-organized, reliable, and well-documented data products.

Key Responsibilities

• Architect, build, and fine-tune the comprehensive medallion framework (Bronze, Silver, Gold) within Microsoft Fabric components such as Lakehouse, Warehouse, and Dataflows Gen2.
• Apply best engineering practices including version control, continuous integration and delivery (CI/CD), and Git-based recovery processes.
• Create a reusable API-driven integration layer that connects various source systems including Premier, Salesforce, Oracle Primavera Cloud, and Azure SQL.
• Develop robust ingestion mechanisms for handling both structured and semi-structured data through REST APIs.
• Prepare and optimize curated datasets for AI and machine learning applications like model training, feature extraction, pricing intelligence, and demand forecasting.
• Enforce strict data quality controls between architecture layers with automated monitoring, alerting, and data lineage tracing.
• Ensure data governance, including residency compliance, tenant-specific security, and role-based access permissions.
• Collaborate closely with analysts and business stakeholders to convert requirements into scalable and efficient data solutions.
• Produce operational runbooks, detailed documentation, and data quality scorecards to support ongoing maintenance and transparency.

Qualification and Experience

• Possess a Bachelor’s degree in Computer Science, Data Engineering, or a related discipline; a Master’s degree is advantageous.
• Bring over a decade of comprehensive experience in data engineering roles.
• Demonstrated proficiency with Microsoft Fabric and practical experience preparing data for AI use cases.
• Strong technical command of SQL, Python, Apache Spark, Power Query, RESTful APIs, Git version control, and Azure services such as Functions, Service Bus, Key Vault, and Blob Storage, including Infrastructure as Code practices.
• Industry experience in construction, contracting, fit-out, or engineering design sectors is highly valued, along with familiarity with systems like Salesforce, Premier, or Oracle Primavera Cloud.

Certifications (Preferred)

• Microsoft Certified Fabric Data Engineer Associate (DP-700)
• Microsoft Certified Fabric Analytics Engineer Associate (DP-600)
• Microsoft Certified Azure Administrator Associate (AZ-104)