C

Lead Data Engineer

Compare the Market Australia

Brisbane, Queensland, Australia · Full Time

Be the first to apply

Experience
10+ yrs
Salary
—
Openings
1
Posted
2 weeks ago
Work mode
In office
Education
Bachelor's degree in Computer Science, IT, Analytics or related discipline
Eligibility
Applicants must have unrestricted Australian work rights; sponsorship is not available.
Resume
Required to apply

Where you'll work

Sign in to tell us what does and doesn't work for you here — it sharpens every match we show you.

Job description

About the Company

Compare the Market Australia and iSelect are reputable brands focused on assisting millions of Australians in comparing and purchasing personal finance and household products like insurance, energy, and loans. Combined, these brands deliver a seamless and customer-centric experience to help users find products tailored to their needs.

Key Responsibilities

As a Lead Data Engineer reporting to the Data Engineering Manager, you will spearhead technical leadership for Compare the Market's Data and Analytics platforms. Your role involves shaping the enterprise data technology stack, setting engineering standards, and deploying scalable, secure, and highly performant data solutions that enable reporting, analytics, AI initiatives, and business innovation.

You will function as the foremost technical expert for data engineering, overseeing the design, governance, and optimization of cloud-based data warehouses, lakehouses, and AI-enabled platforms. Collaborating with engineers, architects, analysts, and business stakeholders, you will champion modern engineering methodologies, platform automation, DataOps, MLOps, and AI engineering to maintain a robust and future-ready data ecosystem aligned with strategic goals.

Key duties include maintaining the technical health, security, performance, scalability, and continuous improvement of the data platform landscape. You will foster a culture of engineering excellence, innovation, and knowledge sharing and provide mentorship and governance for the Data and Analytics team to ensure consistent delivery of high-quality data products and services.

Responsibilities

  • Provide architectural guidance and technical leadership across the Data & Analytics function, ensuring alignment with enterprise architecture, security, scalability, and organizational objectives.
  • Manage and enhance the analytics ecosystem, including cloud data platforms, lakehouse environments, data integration, governance, and AI readiness.
  • Lead development of scalable data products, advanced analytics, and generative AI-enabled solutions that promote self-service insights and future AI capabilities.
  • Promote engineering excellence using standards, frameworks, DataOps, SecOps, continuous integration/continuous deployment (CI/CD), infrastructure as code, automated testing, observability, and automation.
  • Assure the platform’s reliability, security, governance, and compliance through effective data quality controls, lineage tracking, metadata management, access control, monitoring, and privacy safeguards.
  • Mentor technical teams and business stakeholders to drive innovation, platform optimization, adoption of emerging technologies, and advancement of data and AI maturity.

What We Offer

  • A collaborative and supportive culture where your ideas are valued, and personal growth is encouraged.
  • An inclusive workplace driven by core values of Drive, Noble Intent, and All Together.
  • Engagement in meaningful work with recognition programs including quarterly awards.
  • An active social community hosting annual celebrations, family events, and regular gatherings.
  • Flexible work options, such as working one day per week from home or a 9-day fortnight schedule.
  • Additional leave benefits including 'ME' leave, Volunteer leave, leave purchase options, and a Return To Work bonus for parents.
  • Complimentary coffee on Mondays.

Qualifications

  • Bachelor's degree or higher in Computer Science, Analytics, Information Technology, or a related field.

Experience and Skills

  • Over 10 years of professional involvement in Data Engineering, Analytics, Software Engineering, or Business Intelligence, including leadership of large-scale enterprise data programs.
  • Demonstrated expertise designing, building, and managing cloud-based data platforms on AWS, Azure, or Google Cloud Platform (GCP), encompassing data warehouses, lakehouses, and modern data ecosystems.
  • Strong skills in data architecture and platform design, working with Data Warehouses, Data Lakes, Lakehouses, Data Mesh, Medallion Architectures, and other scalable enterprise data solutions.
  • Advanced proficiency with technologies such as Databricks, Snowflake, Microsoft Fabric, Synapse, Redshift, BigQuery, Spark, streaming frameworks, and cloud-native data services.
  • Proven ability implementing best engineering practices like DataOps, SecOps, CI/CD, Infrastructure as Code (IaC), DevOps, automated testing, observability, and platform automation.
  • Deep understanding of enterprise data governance including data quality, metadata management, lineage, cataloging, master data, security, privacy, access controls, auditing, and compliance with regulations.
  • Experience leading data integration, automation, and platform modernization projects featuring APIs, event-driven designs, reusable frameworks, workflows orchestration, and enterprise application integration.
  • Comprehensive knowledge in designing infrastructure supporting AI-centric solutions on Enterprise Data Warehouses (EDWs), involving Generative AI, retrieval-augmented generation (RAG), vector search, embeddings, AI agents, intelligent automation, MLOps, and AI governance concepts.
  • Track record defining technology roadmaps, setting engineering standards, developing architecture patterns, evaluating emerging AI models, and guiding strategic platform investments to boost scalability, reliability, security, performance, and cost efficiency.
  • Demonstrated leadership and mentoring ability, influencing stakeholders, cultivating engineering teams, directing technical strategies, and driving continuous enhancement in Data & Analytics functions and business-as-usual initiatives.

Additional Information

This position is open exclusively to individuals holding unrestricted Australian work rights; sponsorships are not provided. The company is an equal opportunity employer championing diversity and privacy laws compliance. Personal information collected during the application process is confidential and used only for recruitment purposes, stored securely per Australian data protection legislation, and not distributed externally without consent.

Level

Lead

Minimum education

Bachelor's Degree

How they work

Teamwork & Collaboration Leadership Creativity Relationship Building

Leave it if you'd like a reply — we won't use it for anything else.

Click to browse, drag & drop, or paste a screenshot

PNG, JPG, GIF, MP4, WebM, MOV · Max 20MB each · Up to 5 files

Online · instant AI help
Broxer