Data Engineer

Role Overview

Join Creditsafe Group as a Data Engineer in Cardiff, Wales to actively contribute to a cutting-edge codebase, including peer code reviews, and design innovative data processing platforms. You will work closely on building a distributed event-driven architecture that handles metadata using Python, Spark, Airflow, Iceberg, DynamoDB, AWS Glue, and Amazon S3.

Key Responsibilities

• Design and develop a metadata-driven, event-based data processing platform leveraging a broad range of technologies such as Python, Spark, Airflow, Iceberg, DynamoDB, AWS Glue, and S3.
• Integrate Creditsafe’s AWS data lake seamlessly with external data platforms like Snowflake and Databricks.
• Create and scale efficient data pipelines that combine batch and streaming methods to achieve sub-100 milliseconds latency in data serving.
• Play a vital role in the architecture, coding, and deployment phases of the company’s business-critical systems.
• Analyze domain-specific data to offer insights and suggest improvements for existing products.
• Develop cutting-edge AI solutions using methods like semantic search, multi-channel processing (MCP), and agentic workflows.
• Utilize advanced AI tools such as Perplexity, Cursor, and CoPilot to enhance various facets of the engineering workflow.

Additional Information

This position is for an experienced engineer passionate about leveraging modern data architectures and AI to create impactful solutions. Note that placement charges apply for this opportunity.