Azure Data Engineer

About MetLife

MetLife is a leading global financial services organization with operations spanning over 40 countries, delivering insurance, annuities, employee benefits, and asset management. Established in 1868, the company holds significant market positions in the US, Japan, Latin America, Asia, Europe, and the Middle East. Recognized on the Fortune 500 list and the Dow Jones Sustainability Index, MetLife fosters an inclusive, purpose-driven work environment focused on creating a confident future for people worldwide. The company is committed to equal employment opportunities and diversity.

Role Overview

The Azure Data Engineer will be responsible for designing, constructing, and maintaining large-scale cloud data platforms. The position demands expert knowledge in Azure technologies, Databricks, Spark, Lakehouse frameworks, and contemporary data pipeline development. Collaboration with engineers, business teams, and DevOps members is essential to provide secure, dependable, and analytics-enabled data systems in an Agile setting.

Key Responsibilities

• Design and deliver scalable data lakehouse architectures leveraging Azure Data Lake Storage, Azure Databricks, Delta Lake, Unity Catalog, and Medallion approaches.
• Develop curated data products, data marts, and datasets tailored for data analysts, scientists, and reporting units.
• Create reliable batch and near-real-time data pipelines using Databricks, Apache Spark, PySpark, SQL, Azure Data Factory, Synapse, and related Azure tools.
• Construct ingestion frameworks accommodating structured, semi-structured, and unstructured data while ensuring quality, lineage tracking, validation, observability, and reusable designs.
• Apply techniques in data modeling, partitioning, schema evolution, performance tuning, cost efficiency, error management, and operational preparedness.
• Implement CI/CD pipelines for data workflows with Azure DevOps or GitHub Actions, incorporating automated testing, deployment, release management, and secure engineering principles.
• Utilize Entra ID, RBAC, secrets management solutions like Key Vault and Managed Identity, and governance strategies to safeguard data platforms.
• Adopt AI-assisted tools such as GitHub Copilot, Microsoft Copilot, and Databricks Assistant to boost development speed, ETL delivery, testing, documentation, troubleshooting, and productivity.

Candidate Profile

• Proficient in Azure Databricks, Azure Synapse Analytics, Azure Data Factory, PySpark, Spark SQL, and Azure Data Lake Storage Gen2.
• In-depth understanding of Delta Lake technologies, Lakehouse architecture, Unity Catalog, governance mechanisms, Medallion architecture, CDC, SCD, data quality, reconciliation, and lineage frameworks.
• Advanced SQL skills across diverse platforms including Oracle, SQL Server, PostgreSQL, and cloud-based data services.
• Preferred experience with Informatica PowerCenter, primarily in migration projects.
• Familiarity with version control and DevOps tools such as Git, GitHub, Azure DevOps; expertise in CI/CD, automated deployment, secure code practices, monitoring, and operational dashboard creation.
• Strong knowledge of Azure security components including Key Vault, Managed Identity, Entra ID, and role-based access control models.

Technical Environment

• Data Engineering: Azure Databricks, Azure Data Factory, Azure Synapse, ADLS Gen2, Python, PySpark, SQL, Spark SQL, ETL/ELT orchestration, metadata management, and data modeling.
• Lakehouse Architecture: Delta Lake with Bronze, Silver, and Gold layers, schema evolution, Slowly Changing Dimensions (SCD), reusable ingestion frameworks, and optimization techniques.
• DevOps and Quality Assurance: Git, GitHub, Azure DevOps, CI/CD, automated testing frameworks, release pipelines, SonarQube, secure coding standards, Azure Monitor, Application Insights, logging, alerting, and pipeline health monitoring.

Eligibility

Applicants must be graduates from any field.