Big Data Engineer - Python Spark Developer
Chennai, Tamil Nadu, India (Hybrid) · Full Time
Be the first to apply
- Experience
- 3–12 yrs
- Salary
- —
- Openings
- 1
- Posted
- 1 day ago
- Work mode
- Hybrid
- Education
- B.Tech / B.E.
- Eligibility
- Candidates who have completed B.Tech or B.E. in any specialization are eligible to apply.
- Resume
- Required to apply
Where you'll work
Sign in to tell us what does and doesn't work for you here — it sharpens every match we show you.
Job description
About Tech Mahindra
Tech Mahindra delivers technology consulting and digital transformation solutions to numerous global enterprises spanning various industries. With a workforce exceeding 149,000 professionals across more than 90 countries serving over 1,100 clients, Tech Mahindra offers comprehensive services including consulting, IT, enterprise applications, business process and engineering services, network solutions, customer experience design, as well as AI, analytics, and cloud infrastructure services. Notably, it is the first Indian firm globally recognized with the Sustainable Markets Initiatives Terra Carta Seal for advancing climate and nature-positive initiatives.
Job Responsibilities and Role
- Develop and maintain efficient, reusable, and reliable Python code within Big Data ecosystems.
- Engage in coding, testing, debugging, and troubleshooting across the software development lifecycle as a Senior Big Data Developer.
- Utilize Python, PySpark, Scala, and Spark technologies for data pipeline creation and ETL/ELT processes.
- Build automation tools and accelerators to facilitate data migration and pipeline efficiency.
- Operate and manage Big Data toolsets like Hadoop, HDFS, Hive, Kafka, Spark, and Bash scripting, alongside SQL usage.
- Leverage frameworks and libraries proficiently, including experience with data manipulation libraries such as Pandas and Polars.
- Participate in developing and managing data pipelines and enterprise data warehouse solutions.
Requirements and Skills
- Proven professional experience ranging from 3 to 12 years as a Python Data Engineer or Big Data Developer.
- Strong command of Python and Spark programming with solid understanding of Python concepts.
- Hands-on familiarity with PySpark and data manipulation libraries such as Pandas and Polars.
- Experience with Java and SQL programming languages.
- Knowledge of job orchestration tools like Airflow or Control M is advantageous.
- Understanding and experience in developing ELT/ETL pipelines within Hadoop environments.
- Exposure to reporting tools such as Tableau, Dataiku, MicroStrategy is beneficial.
- Familiarity with Reference Data Management and Enterprise Data Warehouse systems is a plus.
- Experience with Azure DevOps is desirable.
Work Location and Conditions
This role is based in Chennai (Sholinganallur) or Bangalore (RMZ Ecoworld, Bellandur) operating in a hybrid work mode with three days onsite per week as per company policy.
Education Eligibility
Applicants must possess a Bachelor of Technology (B.Tech) or Bachelor of Engineering (B.E.) degree in any discipline.
Minimum education
Bachelor's Degree