Data Engineer (Cloud & Analytics Platform)

Role Overview

You will be responsible for designing and maintaining scalable data infrastructure to empower analytics, machine learning, and business intelligence initiatives. This includes building reliable data pipelines and cloud-based platforms, ensuring adherence to data quality, governance, and operational excellence.

Key Responsibilities

• Create, develop, and sustain ETL/ELT workflows in both cloud and on-premise environments, incorporating internal and external data sources, including compliant public data ingestion.
• Construct and optimize scalable data models like star and snowflake schemas to facilitate reporting and analytical tasks.
• Maintain data accuracy, completeness, and accessibility through validation, cleansing, transformation, and monitoring controls.
• Deploy and manage Microsoft Azure cloud services such as Azure Data Factory, Data Lake Storage, Azure SQL, and associated components.
• Handle infrastructure setups including compute, storage, networking, identity management, and security to assure scalable and dependable platform functionality.
• Develop CI/CD pipelines, automate deployments, and implement monitoring, logging, and alerting systems to uphold performance standards and SLAs.
• Assist in developing interactive dashboards and reports using Power BI, ensuring smooth data flow and integration.
• Work closely with business teams to translate their requirements into functional data solutions and visualizations.
• Collaborate alongside Data Scientists to operationalize and monitor machine learning models and embed AI/ML capabilities within data processes and applications.
• Adhere to data governance protocols including security, compliance, access control, and data lineage while applying software engineering best practices like version control and code reviews.
• Drive ongoing improvements to data platform standards to enhance operational efficiency, scalability, and cost-effectiveness.

Candidate Profile

• Bachelor's degree in Computer Science, Computer Engineering, or related disciplines.
• 2 to 3 years of relevant experience in data or software engineering involving data platforms, cloud technologies, and analytics.
• Strong analytical capabilities, adept problem-solving, and effective troubleshooting skills.
• Effective communicator with collaborative mindset for working across technical and business units.
• Detail-oriented, self-driven, capable of prioritizing multiple assignments in fast-paced settings.
• Eagerness to acquire and adopt new technologies and methodologies continually.

Technical Expertise Required

• Knowledge of data warehousing principles, data modeling, database tuning, and ETL/ELT development using Python or ETL tools such as SSIS or Informatica.
• Experience with relational databases including Microsoft SQL Server and Oracle; acquainted with big data platforms like Spark and Databricks.
• Proficient in Python for data manipulation, engineering tasks, and basic API creation.
• Hands-on experience with Microsoft Azure tools: Data Factory, Data Lake Storage, and Azure SQL.
• Familiarity with CI/CD pipelines, DevOps methodologies, and automation using Azure DevOps.
• Understanding of real-time/streaming data processing concepts plus skills in monitoring and performance tuning for data systems.
• Competence in Power BI including DAX and Power Query for report generation and dashboard design.
• Basic knowledge of machine learning processes and tools including Scikit-learn and Azure Machine Learning.

Additional Information

Only candidates shortlisted for further evaluation will be contacted.