Data Engineer

About the Role

We are collaborating with a leading Australian multi-location retail and franchise network to establish their data and AI capabilities from the ground up. This role is a practical Data Engineer position within a newly created Data & Insights team, tasked with constructing and managing pipelines, models, and reporting infrastructures that enable commercial decision-making across a vast national network.

You will operate within the central Digital, Data, and Technology division, integrating data from multiple point-of-sale systems, ecommerce channels, and marketing platforms into a governed Snowflake environment. The goal is to convert raw data into reliable, analytics-ready datasets accessible to the entire organisation.

Key Responsibilities

• Develop and maintain ELT pipelines to import data from source systems such as legacy ERPs/iSeries, enterprise ecommerce, and marketing platforms into Snowflake.
• Create and maintain dbt models applying a medallion architecture and rigorous modelling standards.
• Design dimensional data models—including customer, product, location, and inventory dimensions, along with related fact tables—to underpin trustworthy reporting and future AI applications.
• Utilise Azure Data Factory and Azure Data Lake Storage (ADLS) for ingestion workflows and assist in evaluating and deploying iPaaS tools.
• Participate in AI projects, including integration of Large Language Models (e.g., Claude, ChatGPT, Gemini) and design of autonomous AI workflows enhancing platform functionality.
• Identify and resolve data quality challenges at the source level, such as missing values, inconsistent keys, and duplicates, by implementing robust validation and cleansing rules.
• Support Power BI reporting by ensuring datasets are optimally modelled, performant, and well documented.
• Configure iPaaS integrations to link operational systems, automate data transfer, and decrease manual interventions.
• Apply best practices in CI/CD, version control, and testing to the data platform.
• Collaborate with data leaders, business analysts, and stakeholders across ecommerce, marketing, and retail operations to convert complex needs into dependable data products.

Candidate Requirements

• Between five and eight years of experience in data engineering, with proven expertise in pipeline development, data quality assurance, and master data management.
• Proficiency in SQL and Python, capable of authoring clean, production-quality code; familiar with using Python through Snowpark for complex tasks like entity resolution, probabilistic matching, and data quality profiling.
• Strong experience with Snowflake and dbt, including usage of testing frameworks and documentation tools.
• Demonstrated skills in master data management, entity resolution, and customer data matching employing deterministic and probabilistic deduplication techniques.
• Experience creating and managing reference data sets such as hierarchies, lookup tables, and classification schemes.
• Hands-on expertise with Azure Data Factory and ADLS for data ingestion and workflow orchestration.
• Direct experience configuring and delivering integrations using iPaaS technology.
• In-depth knowledge of data quality profiling, validation rule development, and monitoring within a layered data warehouse architecture.
• Strong familiarity with dimensional modelling concepts (star schemas, Kimball methodology) and ELT design patterns.
• Capability to structure efficient and performant datasets for business intelligence tools, preferably Power BI.
• Comfort handling legacy or complex source systems while effectively communicating technical trade-offs to non-technical stakeholders.
• Experience with Git, continuous integration/continuous deployment (CI/CD), and agile development methodologies.
• A proactive, delivery-focused attitude: clearly defining project scope, building iteratively, and thriving within a small, fast-paced team that is establishing a new platform.

Preferred Qualifications

• Previous experience working in complex multi-site retail, high-value product domains, or large-scale ecommerce enterprises.
• Knowledge of Product Information Management (PIM), ERP systems, or management systems for retail/franchise operations.
• Experience extracting data from legacy ERP platforms like AS400/iSeries.
• An avid interest in data governance and advancements in artificial intelligence applications.