Data Engineer (Cloud & Analytics Platform)

Job Overview

The role entails designing, implementing, and maintaining reliable and scalable data systems that underpin analytics, machine learning, and business intelligence initiatives. This includes developing data pipelines, leveraging cloud data platforms, supporting AI/ML projects, and ensuring high standards of data quality, governance, and operational effectiveness.

Key Responsibilities

• Create, build, and sustain ETL/ELT workflows across both cloud and local data centers, integrating diverse internal and external datasets with compliance considerations for public data ingestion.
• Develop and fine-tune robust data models, such as star and snowflake schemas, to enable insightful analytics and reporting functions.
• Maintain data accuracy, integrity, and availability by performing validation, cleansing, transformations, and deploying monitoring mechanisms.
• Deploy and manage cloud data services within Microsoft Azure including Azure Data Factory, Data Lake Storage, and Azure SQL Database.
• Handle infrastructure set-up and management concerning computing resources, storage, networking, identity access, and security to guarantee platform dependability and scalability.
• Adopt and manage CI/CD workflows, automate deployments, and establish monitoring, logging, and alert systems to uphold optimal performance and reliability within service level agreements.
• Facilitate the creation and improvement of interactive dashboards and visual reports using Power BI, ensuring seamless linkage between data ingestion and reporting layers.
• Work collaboratively with business teams to gather requirements and translate these into practical data-centric solutions and visual analytics.
• Partner with Data Scientists to deploy, maintain, and oversee machine learning model production, embedding AI/ML capabilities into data workflows and applications.
• Enforce data governance policies, security standards, compliance requirements, access permissions, and data traceability while adhering to best practices in software engineering including version control and code reviews.
• Drive ongoing enhancements in platform standards, operational efficiency, scalability, and cost-effectiveness.

Candidate Requirements

• Educationally, hold a bachelor's degree in Computer Science, Computer Engineering, or a closely related discipline.
• Have between 2 to 3 years of relevant work in data engineering, software development, or associated roles with exposure to cloud data platforms and analytics environments.
• Demonstrate strong analytical, problem-solving, and debugging capabilities.
• Exhibit effective communication and teamwork skills to collaborate across cross-functional technical and business units.
• Show self-drive, attention to detail, and the ability to prioritize multiple tasks within a fast-paced setting.
• Possess enthusiasm for learning emerging technologies, tools, and approaches.

Technical Proficiencies

• Experience in data warehousing concepts, designing data models, enhancing database performance, and crafting ETL/ELT pipelines using Python or tools such as SSIS or Informatica.
• Proficient with relational databases like Microsoft SQL Server and Oracle, and have familiarity with big data platforms like Apache Spark and Databricks.
• Python programming adeptness for data transformation, engineering, and foundational API creation.
• Hands-on expertise with Microsoft Azure services, notably Azure Data Factory, Data Lake Storage, and Azure SQL solutions.
• Knowledge of CI/CD methodologies, DevOps culture, and automation utilities such as Azure DevOps.
• Understanding of real-time data processing and streaming paradigms, including system monitoring and performance tuning.
• Experience in developing reports and dashboards with Power BI, utilizing DAX and Power Query.
• Basic comprehension of machine learning lifecycle and tooling, including Scikit-learn and Azure Machine Learning.

Additional Information

Only applicants selected for further consideration will be contacted.