Data Engineer + Business Intelligence Specialist

Overview

We seek an accomplished Data Engineer and Business Intelligence expert with over 7 years of hands-on experience in designing and implementing scalable data solutions. Proficiency in cloud data platforms, Python programming, SQL, ETL processes, Big Data technologies, and BI tools is essential. The candidate will contribute to creating effective data pipelines, leveraging AWS services, crafting data transformations, and delivering insightful visual analytics.

Core Responsibilities

• Design and maintain large-scale, efficient data engineering and processing architectures.
• Develop robust ETL/ELT workflows utilizing AWS Glue and related tools.
• Create data transformation pipelines using Python and PySpark.
• Utilize AWS cloud services to establish reliable and scalable data solutions.
• Integrate diverse data sources into cohesive data pipelines.
• Craft and optimize SQL queries and PostgreSQL database structures.
• Work with Snowflake and DBT tools for data modeling and analytical processing.
• Construct and sustain data models and analysis-ready datasets.
• Design dashboards and reports with Tableau, Qlik, or similar visualization platforms.
• Collaborate closely with business and tech teams to align analytics with requirements.
• Adhere to best practices regarding data engineering, quality, performance, and development.

Technical Skills Required

• Minimum 7 years of data engineering and development experience.
• Extensive knowledge of scalable data pipeline architecture and processing solutions.
• Proficient understanding of ETL/ELT processes, data transformations, and modeling concepts.
• At least 5 years of working with SQL and relational databases, particularly PostgreSQL.
• Strong Python programming skills with a minimum of 5 years of data-related experience.
• Experience building ETL pipelines with AWS Glue and handling Big Data frameworks such as PySpark.
• At least 3 years of experience using AWS cloud services, including Lambda, ECS, Step Functions, API Gateway, RDS, SQS, and DynamoDB.
• Competency in Business Intelligence tools like Tableau, Qlik, or equivalent; preference given for ThoughtSpot experience.
• Hands-on familiarity with Snowflake data warehouse and DBT for data transformation and modeling.

Desirable Skills

• AWS Data Engineer certification is an advantage.
• Knowledge of Git and continuous integration/continuous deployment (CI/CD) methods.
• Experience with GitLab or comparable CI/CD platforms.
• Understanding of Infrastructure as Code (IaC) concepts and tools like Terraform.
• Background in pharmaceutical domain considered a plus.
• Exposure to AI-assisted coding tools such as VS Code Copilot or similar.

Candidate Profile

The ideal candidate is an adept data engineer with comprehensive expertise in AWS services, Python, SQL, and BI platforms, capable of managing the entire data lifecycle from ingestion through visualization. Effective communication, problem-solving capabilities, collaboration, and a sense of ownership are integral attributes.

Eligibility

Applications are open to all graduates.