Senior Data Platforms Engineer

About Simple Machines

Simple Machines is a globally independent technology consultancy with offices in Sydney, New Zealand, London, Poland, and San Francisco. We specialize in designing and building cutting-edge data platforms, intelligent systems, and custom software solutions that integrate Data Engineering, Software Engineering, and AI. Our clients include enterprises, scale-ups, and government agencies, whom we help transform complex, high-value data into actionable products, platforms, and insights. Our motto: We engineer data to life™.

Position Overview

We are seeking a hands-on Senior Data Platforms Engineer to architect and develop new data platforms, real-time data pipelines, and data products for clients dedicated to leveraging data effectively. This role involves close client collaboration and working within compact, highly skilled teams. This is not a purely architectural or support role, but one focused on delivering impactful data solutions.

Key Responsibilities

• Architect and deploy modern cloud-native data platforms leveraging Databricks, Snowflake, AWS, and GCP technologies.
• Implement advanced architectural concepts such as data mesh, data products, and data contracts.
• Deeply integrate with client systems to enable scalable, user-centric data access.
• Develop and optimize both batch and streaming data pipelines using technologies like Kafka, Flink, Kinesis, and Pub/Sub.
• Manage workflow orchestration tools including Airflow, Dataflow, and Glue.
• Process and transform large-scale data using Spark and Flink to ensure production-ready system performance.
• Handle various storage solutions across relational, NoSQL, and analytics databases such as Postgres, BigQuery, Snowflake, Cassandra, and MongoDB.
• Optimize data storage formats and enhance data access efficiency through formats like Parquet, Delta, ORC, and Avro.
• Ensure data solutions are secure and compliant by implementing security by design principles without compromising developer productivity.
• Embed strong governance frameworks throughout data platforms.
• Engage directly with clients to understand business challenges and translate them into practical engineering solutions.
• Serve as a trusted technical advisor and drive impactful outcomes beyond just code delivery.

Candidate Qualifications

• Proficient in advanced Python and SQL programming.
• Demonstrated experience with Apache Spark and contemporary data platforms such as Databricks or Snowflake.
• Strong understanding and hands-on experience with cloud data services, especially AWS or GCP.
• Proven track record of building and managing large-scale, production-grade data pipelines.
• Expertise in data modeling and sound architectural judgment for data platform design.
• Comfortable working with diverse storage technologies and data formats.
• Experience with infrastructure-as-code tools including Terraform or Pulumi.
• Familiarity with CI/CD tools, for instance GitHub Actions, ArgoCD.
• Knowledge of data quality and testing frameworks such as dbt, Great Expectations, or Soda.
• Additional advantage if skilled in JVM-based or systems programming languages like Scala, Java, Go, or Rust.
• Experience with graph or specialized databases like Neo4j or Cassandra is beneficial.
• Background in consulting or professional services and a mindset focused on delivery and agile teamwork.
• Capability to engage with stakeholders effectively to generate solutions and influence outcomes.

Benefits

• Opportunity to work on challenging, impactful problems.
• Chance to build modern platforms rather than maintain legacy systems.
• Collaborative environment surrounded by experienced senior engineers.
• High autonomy, meaningful influence, and substantial room for career growth.