Data Engineer

Job Overview

Join BCT Consulting Private Limited as a Data Engineer responsible for architecting, building, and managing extensive data pipelines using Python and PySpark technologies. Collaborate with diverse teams to understand requirements and provide efficient, scalable data solutions. Your role will focus on creating robust data warehousing architectures on AWS and maintaining high-quality data through effective testing and monitoring.

Key Responsibilities

• Architect and sustain large-scale data pipelines utilizing Python and PySpark.
• Work closely with cross-departmental teams to gather data requirements and deliver optimal solutions.
• Design and implement scalable data warehousing systems on the AWS cloud platform.
• Achieve and maintain data integrity by developing thorough testing frameworks and continuous monitoring dashboards.
• Engage in code reviews to uphold best coding practices and standards.

Qualifications and Skills

• Between 4 to 9 years of practical experience in Data Engineering, specifically with Data Warehousing expertise.
• Advanced knowledge of Python programming coupled with hands-on experience in PySpark.
• Proficient in designing and building ETL workflows for large datasets, particularly leveraging AWS tools like S3, Lambda, and Step Functions.
• Solid understanding of SQL including query optimization techniques to enhance performance.

Eligibility

Applications are welcome from any graduates meeting the experience criteria.