Databricks Data Engineer (Data & AI)

Company Overview

At Deloitte in Aotearoa, you join a team driven to make meaningful impacts by harnessing curiosity, passion, and collaboration with talented individuals. The firm values inclusion and respects the cultural uniqueness of New Zealand, integrating principles such as Te Tiriti o Waitangi and fostering a welcoming and supportive workplace environment.

Role Summary

We are seeking a skilled Databricks Data Engineer to contribute to our expanding AI and Data Engineering group. This role focuses on designing, constructing, and maintaining scalable data and AI solutions using the Databricks platform. Ideal candidates are enthusiastic about contemporary data engineering, adept at overcoming intricate technical and business challenges, and eager to employ AI-aided engineering techniques for faster and smarter outcome delivery.

Key Responsibilities

• Create and manage scalable data and AI solutions with Databricks technologies.
• Develop robust data ingestion and transformation pipelines including ELT/ETL processes.
• Craft reusable data transformations using Lakeflow Declarative Pipelines.
• Architect and implement Lakehouse structures supporting diverse analytics and AI use cases.
• Build dimensional, curated, and feature-ready models to facilitate reporting, analytics, machine learning, and generative AI applications.
• Integrate Databricks with cloud services, APIs, databases, streaming systems, and enterprise software.
• Implement automated CI/CD workflows and adopt modern DevOps methodologies for deployment.
• Manage orchestration of data workloads via Lakeflow Jobs, Azure Data Factory, Airflow, or comparable tools.
• Ensure adherence to security, governance, data quality, lineage, and operational monitoring standards.
• Optimize Databricks environments for enhanced performance, scalability, reliability, and cost effectiveness.
• Diagnose and resolve complex issues in platform and pipeline operations across development and production.
• Collaborate with architects, data scientists, ML engineers, analysts, and business teams to translate requirements into viable technical designs.
• Contribute actively to engineering best practices, reusable components, and technical documentation with continuous process improvements.
• Utilize AI-driven tools like Genie Code, GitHub Copilot, and Claude to expedite coding, testing, troubleshooting, documentation, and reviews.
• Identify automation opportunities using AI to minimize repetitive tasks and hasten development cycles.

Qualification and Skills

• Proven hands-on expertise with Databricks platform.
• Experience designing and implementing modern Lakehouse architectures.
• Profound knowledge of data engineering fundamentals including modelling, warehousing, and distributed computation.
• Capability to build both batch and streaming data pipelines.
• Familiarity with orchestration tools such as Lakeflow Jobs, Azure Data Factory, or Airflow.
• Understanding of Azure and/or AWS cloud data services.
• Competence with Git version control, CI/CD pipelines, and modern software engineering standards.
• Advantageous to have Python programming skills applied in data engineering projects.
• Exposure to infrastructure-as-code tools like Terraform or similar is beneficial.
• Strong analytical capabilities and problem-solving skills.
• Effective communication skills with experience engaging stakeholders.

Preferred Attributes

• Experience with Generative AI, Machine Learning, or MLOps implementations on Databricks.
• Knowledge of Databricks Declarative Automation Bundles.
• Possession of relevant Databricks certifications such as Data Engineer Associate or Professional.

AI-Assisted Engineering Approach

Embrace an AI-enhanced engineering mindset by using tools like Genie Code, GitHub Copilot, and Claude to accelerate development, automate repetitive tasks, elevate code quality, and advance efficiency—all while ensuring security, maintainability, and engineering excellence.

What Success Looks Like

The successful candidate takes full ownership of data engineering challenges from design through production, prioritizing solutions that are scalable, secure, automated, maintainable, and high quality. Comfort with cross-functional collaboration and continuous improvement is essential, alongside a keen interest in leveraging AI to enhance workflows.

Why Join Deloitte?

• Engage with premier clients and exciting projects in New Zealand.
• Work on large-scale, modern Databricks Lakehouse platforms.
• Deliver integrated solutions spanning data engineering, AI, analytics, machine learning, and generative AI.
• Tackle real-world complex engineering problems.
• Expand expertise in AI-assisted development and modern engineering disciplines.
• Collaborate with experienced professionals including data engineers, architects, and technology leaders.
• Contribute to frameworks and practices advancing modern data platform delivery.

Additional Information and Benefits

• Flexible work arrangements aligned with personal and client needs.
• Support for professional certifications and paid memberships.
• Subsidised health insurance and income protection coverage.
• Generous annual leave entitlement of 22 days.
• Opportunities for social events, community volunteering, and wellbeing initiatives.
• Commitment to diversity, inclusion, and equal opportunity employment.

Important Notes

Beware of phishing scams; all legitimate communications will originate from Deloitte official domains. Verify suspicious contacts before engagement.