Lead Data Software Engineer

Position Overview

Our client in Dublin is seeking a seasoned Lead Data Engineer to take on a pivotal technical leadership role involved in designing, developing, and advancing cloud-based data platforms. This role requires hands-on architectural expertise and deep experience in Python, AWS, and enterprise-scale data engineering solutions.

Key Responsibilities

• Lead the design, architecture, and deployment of scalable and secure data engineering solutions that offer high performance.
• Develop cloud-native data platforms using AWS technologies coupled with Python programming.
• Provide technical leadership and architecture guidance across various data engineering projects while actively engaging in the solution delivery process.
• Create robust ETL/ELT pipelines to facilitate large-scale data ingestion, transformation, and integration.
• Champion best practices around data architecture, data governance, security, scalability, and enhancing system performance.
• Work closely with architects, software developers, business stakeholders, and product teams to accomplish enterprise data objectives.
• Research and assess emerging technologies to recommend enhancements to existing data engineering frameworks.
• Mentor engineering teams through technical leadership activities such as code reviews and offering architectural advice, without direct personnel management duties.
• Ensure solutions adhere to enterprise architecture standards, are resilient, and maintainable.

Experience and Qualifications

• At least 10 years of experience in Data Engineering, Data Platform Engineering, or closely related fields.
• Proven track record designing and implementing data platforms at an enterprise level.
• Advanced skills in Python development.
• Extensive experience delivering cloud-based data solutions on AWS platforms.
• Strong background in designing scalable data architectures and data pipelines.
• Comprehensive understanding of data modeling, warehousing, and distributed processing.
• Experience handling complex and large enterprise datasets.
• Excellent communication skills for engaging diverse stakeholders and simplifying technical concepts.
• Strong analytical and problem-solving skills focused on high-quality delivery.

Desired Technical Expertise

• Hands-on experience with AWS Glue, Lambda, EMR, Redshift, S3, Athena, and Step Functions.
• Familiarity with Infrastructure as Code tools like Terraform or AWS CloudFormation.
• Knowledge of orchestration frameworks such as Apache Airflow.
• Experience with container technologies including Docker and Kubernetes.
• Exposure to streaming platforms like Kafka or Kinesis.
• Understanding of CI/CD pipelines and DevOps methodologies.
• Experience working under Agile methodologies.
• Awareness of data governance, security, and compliance best practices.

Education

• Bachelor’s degree or higher in Computer Science, Software Engineering, Information Technology, or related disciplines.
• Relevant cloud or data engineering certifications (AWS Certified Solutions Architect, AWS Certified Data Engineer, or similar) are advantageous.

Working Conditions and Benefits

• Hybrid working arrangement based in Dublin 2.
• Competitive remuneration package around €120,000 base salary plus bonus and comprehensive benefits.
• Bonus linked to performance.
• Pension scheme contributions.
• Comprehensive healthcare package.
• Life insurance coverage.
• Support for professional training and certification.
• Opportunities for career advancement within a growing technology team.