Lead Data Engineer

About the Role

This opportunity is geared towards experienced Data Engineers who meet the eligibility criteria for SC clearance. Our company partners with some of the largest UK enterprises and government entities, offering tailored technology solutions and specialist guidance.

As a Data Engineer, you will develop foundational data platforms and pipelines that empower a variety of data and analytics solutions, including AI and data science applications. Your work will encompass building data warehouses and lakes, crafting data ingestion and access processes, and integrating disparate datasets into cohesive data models for enhanced business insight and machine learning purposes.

This is a hands-on, client-facing position available at senior or lead level. Responsibilities may involve leading teams, steering technical strategy, consulting clients, and tackling complex engineering problems. Occasional on-site presence in London for client meetings will be expected.

Key Responsibilities

• Develop and maintain data storage and pipeline architectures to ingest, transform, and query analytical datasets.
• Build scalable data lakes and data warehouses to support business and analytical requirements.
• Design integrated data models from siloed data sources to facilitate business insights and machine learning workflows.
• Collaborate closely with clients to understand their data challenges and provide effective engineering solutions.
• Lead technical direction and mentor other engineers where applicable.
• Conduct occasional on-site client engagements in the London area on an ad hoc basis.

Candidate Requirements

• Proven expertise with modern data engineering technologies and methodologies, including data storage, pipeline development, and analytical querying.
• Hands-on experience with tools such as Python, Spark, PySpark, SQL, and PowerBI.
• Strong software engineering background including experience with front-end technologies like JavaScript.
• Analytical problem-solving aptitude, able to pragmatically evaluate alternatives and devise efficient solutions.
• Knowledge of designing and constructing clean, maintainable systems.
• Excellent communication and teamwork skills, with a collaborative mindset.
• Eagerness to learn new technologies and continually expand your skillset.

Preferred but Not Required

• Familiarity with cloud platforms including AWS, Azure, or Google Cloud Platform.
• Experience operating within Agile development teams.
• Exposure to vendor tools such as Snowflake or Databricks.
• Knowledge of CI/CD pipelines and automation tooling.

Benefits and Perks

• Starting with 25 days annual leave, increasing by one day per year up to 30 days.
• Generous family leave arrangements.
• Access to company pension plan, private healthcare, and group life insurance.
• Various optional perks including discounted gym memberships and cycle-to-work schemes.
• Structured performance evaluation and feedback for career progression.
• Hybrid remote working supported with occasional office and client visits.
• Employee-run clubs, events, free refreshments, and recreational resources at the office.
• Commitment to inclusive and equitable workplace values, backed by B Corp certification.

Additional Information

Applicants must be eligible for Security Clearance (SC). Please review the government guidance on clearance eligibility prior to applying.

Scott Logic strongly embraces diversity, aiming to cultivate an environment where everyone is free to contribute regardless of background or identity.