Senior Data Engineer

About Numerator

Numerator is reshaping the future of market research by providing leading brands and retailers with unparalleled insights into consumer behavior and key drivers influencing it.

Role Overview

We are looking for an experienced Senior Data Engineer to design, build, and optimize data pipelines managing large-scale data volumes. This role requires expertise in Python programming, Azure Cloud's Databricks platform, DevOps practices, and proficiency in big data frameworks like Apache Spark and PySpark, along with a fundamental understanding of AI and machine learning techniques.

Key Responsibilities

• Follow coding standards and technology protocols established by Numerator.
• Develop automation test suites within Azure DevOps environments.
• Maintain and refine existing automation tests as needed.
• Execute manual, load, and exploratory testing to ensure solution quality.
• Collaborate closely with Business Analysts and Senior Data Developers to consistently meet sprint objectives.
• Participate in planning and estimating sprint tasks and user stories.
• Take ownership and accountability for product delivery processes.

Required Qualifications and Skills

• Between 7 and 10 years of experience in data engineering positions.
• Strong skills in Python programming.
• Hands-on experience with Microsoft Azure Cloud services.
• Familiarity and practical experience with Agile methodologies such as Scrum or Kanban.
• Experience using Apache Spark, PySpark, and Databricks for big data processing.
• Knowledge of DevOps pipelines, preferably with Azure DevOps tools.

Preferred Credentials

• A bachelor's or master's degree in Computer Science, Information Technology, Data Science, or a related discipline.
• Experience working in support-oriented roles.
• Understanding or experience in AI and machine learning methods.
• Relevant certifications in data engineering or related technical fields.