Data Engineer

About the Role

We seek a skilled Data Engineer with over five years of experience to work onsite in locations including Bangalore, Chennai, Hyderabad, Pune, and Kolkata. The successful candidate will be instrumental in designing, building, and maintaining scalable batch and streaming data pipelines to support our analytics and platform engineering teams.

Responsibilities

• Design, develop, and maintain scalable batch and streaming data pipelines.
• Implement distributed data processing frameworks such as Spark, Apache Beam, or Flink.
• Manage workflow orchestration utilizing Apache Airflow.
• Create data transformations leveraging Python, SQL, and dbt.
• Operate and manage data pipelines on cloud platforms, preferably Google Cloud Platform (GCP).
• Ensure high data quality, optimal performance, and reliability of pipelines.
• Collaborate effectively with Analytics and Platform Engineering teams.

Requirements

• Proficient in Python and SQL programming languages.
• Experienced with Apache Airflow for orchestration.
• Hands-on experience with Spark, Apache Beam, or Flink.
• Skilled in using dbt for data transformations.
• Experience working with cloud environments, preferably GCP.
• Knowledge of streaming technologies such as Kafka or Pub/Sub.
• Familiarity with AI-assisted coding tools like GitHub Copilot, Cursor, or Gemini Code Assist.

Preferred Qualifications

• Experience with AI-driven data or machine learning pipelines.
• Background in data quality management and governance.
• Exposure to business intelligence and analytics platforms.