PySpark Developer with AWS
Kolkata, West Bengal, India · Full Time
Be the first to apply
- Experience
- 8–10 yrs
- Salary
- —
- Openings
- 1
- Posted
- 10 seconds ago
- Work mode
- In office
- Education
- Any graduate
- Eligibility
- Candidates with any graduate degree may apply.
- Resume
- Required to apply
Where you'll work
Sign in to tell us what does and doesn't work for you here — it sharpens every match we show you.
Job description
About the Company
Tata Consultancy Services (TCS) is a renowned global IT and business solutions provider with over 612,000 employees across 55 countries and reported revenues of USD 29.08 billion for 2023-24. The company delivers a diverse portfolio of services including Cloud Infrastructure, Cognitive Business Operations, Cybersecurity, Consulting, Enterprise Solutions, Internet of Things/Digital Engineering, and Data & Analytics. Notably, TCS is one of India's largest equal opportunity employers with over 100,000 female associates and consistently recognized as a leader in India's Most Respected Companies rankings.
Job Summary
We are seeking a skilled PySpark Developer with substantial AWS cloud experience to architect, develop, and enhance scalable data-processing systems. The ideal candidate will possess robust hands-on expertise in PySpark, Python programming, Apache Spark, SQL, ETL/data pipeline creation, and AWS services.
Primary Responsibilities
- Design, develop, and sustain large-scale data processing applications using PySpark.
- Create and maintain scalable ETL/ELT data pipelines to facilitate data transformation and integration.
- Utilize various AWS cloud services to support and execute data engineering workloads effectively.
- Craft and improve complex SQL queries for efficient data retrieval and manipulation.
- Enhance PySpark and Spark job performance and scalability through optimization techniques.
- Identify and resolve issues related to data pipelines and uphold data quality standards.
- Collaborate closely with data architects and fellow engineering teams to align solutions with organizational needs.
Key Skills and Requirements
- Strong proficiency with PySpark development.
- Mandatory experience working with AWS cloud services.
- Expertise in Apache Spark framework.
- Solid programming skills in Python.
- Advanced knowledge of SQL for querying and managing databases.
- Proven experience in ETL and overall data engineering practices.
- Competence in developing and maintaining complex data pipelines.
- Familiarity with big data processing concepts and tools.
- Hands-on experience with AWS services including S3, Glue, EMR, Lambda, Redshift, IAM, and CloudWatch.
Eligibility
Open to candidates holding any graduate degree.
Minimum education
Bachelor's Degree