Senior Databricks Engineer

Job Summary

We are searching for a seasoned Databricks expert to lead the advancement of our next-generation platform, define technical best practices, and oversee delivery within our Lakehouse ecosystem.

Responsibilities

• Lead efforts to modernize the Databricks platform, integrating and promoting advanced functionalities.
• Design and implement Lakeflow jobs, pipelines, and scalable solutions within the Lakehouse architecture.
• Deploy and enhance Unity Catalog, DQX, DB Genie, and AI/BI workloads effectively.
• Develop engineering best practices, governance policies, and automation frameworks to ensure operational excellence.
• Mentor technical teams and provide guidance on architectural and delivery decisions across the organization.

Requirements

• At least 5 years of experience in data engineering or platform engineering with extensive expertise in Databricks.
• Practical, hands-on knowledge of Pyspark, Streaming data processing, Databricks Runtime (DBR), Lakeflow, Unity Catalog (UC), DQX, DB Genie, and Delta Lake.
• Familiarity with Azure Data Factory (ADF), Azure Synapse, and practical exposure to Azure DevOps tools.
• Demonstrated leadership in managing large-scale Databricks or Lakehouse transformation initiatives.
• Exceptional problem-solving skills, innovative thinking, and proven ability to lead deliverables.