Databricks Data Engineer (Data & AI)

About the Role

Join Deloitte as a Databricks Data Engineer within our expanding AI & Data Engineering team in Christchurch. This position is suited for individuals enthusiastic about contemporary data engineering, eager to tackle complex technical and business challenges, and excited to adopt AI-driven engineering methods to accelerate and enhance solution delivery.

Key Responsibilities

• Architect, develop, and sustain scalable data and AI solutions leveraging Databricks technology.
• Create reliable data ingestion, ETL/ELT, and processing workflows on Databricks.
• Implement reusable, high-quality data transformations utilizing Lakeflow Declarative Pipelines.
• Design and build Lakehouse architecture solutions.
• Construct dimensional and feature-focused data models supporting analytics, machine learning, and Generative AI initiatives.
• Integrate Databricks with cloud services, APIs, databases, streaming platforms, and enterprise systems.
• Develop automated CI/CD pipelines and follow modern DevOps practices for deploying Databricks solutions.
• Manage workload orchestration using Lakeflow Jobs, Azure Data Factory, Airflow, or equivalent tools.
• Apply controls for security, governance, data quality, lineage tracking, and monitoring.
• Optimize Databricks workloads for efficiency, scalability, reliability, and cost-effectiveness.
• Troubleshoot complex issues across development and production environments.
• Collaborate with architects, analysts, data scientists, ML engineers, and business partners to deliver practical technical solutions.
• Contribute to engineering standards, reusable frameworks, and comprehensive documentation for continuous improvement.
• Utilize AI-based development tools such as Genie Code and GitHub Copilot to enhance coding, testing, troubleshooting, and documentation.
• Identify opportunities to implement AI and automation to reduce manual effort and speed development cycles.

Required Qualifications and Experience

• Proven practical expertise with Databricks technologies.
• Experience designing and implementing Lakehouse architectures.
• Deep understanding of data engineering fundamentals, data modeling, warehousing, and distributed data processing.
• Skilled in building batch and streaming data pipelines.
• Familiarity with orchestration tools such as Lakeflow Jobs, Azure Data Factory, or Airflow.
• Knowledge of cloud data solutions on Azure and/or AWS platforms.
• Competence with Git, CI/CD workflows, and contemporary software engineering methodologies.
• Python experience in data engineering contexts is advantageous.
• Awareness of infrastructure-as-code tools like Terraform.
• Strong analytical and problem-solving capabilities.
• Effective communication and stakeholder engagement skills.

Preferred Experience

• Exposure to Generative AI, machine learning, or MLOps using Databricks.
• Familiarity with Databricks Declarative Automation Bundles.
• Holding certifications such as Databricks Certified Data Engineer Associate or Professional.

AI-Enhanced Engineering Approach

Adopt an AI-driven engineering mindset by employing tools including Genie Code, Claude, GitHub Copilot to expedite development, automate repetitive tasks, and improve code quality. Use critical judgment to ensure outputs are secure, maintainable, and accurate.

What Success Looks Like

You demonstrate capability to independently manage data engineering challenges from design through deployment. You value creating solutions that are functional, well-engineered, scalable, maintainable, secure, and automated. You engage actively with cloud, data, analytic, and AI ecosystems, continuously seeking improvement through reusable patterns, automation, and AI tool utilization. Your approach combines robust technical skills with a focus on delivery and continuous advancement.

Why Work With Us?

• Engage with projects for leading New Zealand companies.
• Work on modern, enterprise-grade Databricks Lakehouse platforms.
• Deliver solutions spanning Data Engineering, Analytics, AI, Machine Learning, and Generative AI.
• Address complex, practical data engineering problems.
• Gain experience in current engineering and AI-assisted development practices.
• Collaborate with expert engineers, architects, platform engineers, and technology leaders.
• Contribute to frameworks and tools that enhance delivery of modern data platforms.

Additional Information

Deloitte New Zealand is dedicated to impactful delivery supported by a strong leadership team and global member firms. We recognize and uphold New Zealand’s unique cultural heritage and strive to foster an inclusive workplace aligned with core values such as care and inclusion. We offer flexible work arrangements, support for professional development, health insurance subsidization, generous leave policies, community volunteering opportunities, and wellness initiatives. Diverse applicants, including women and underrepresented groups, are encouraged to apply even if not all criteria are met.