Senior Data Engineer - Spark, Scala, Azure Databricks
Bengaluru, Karnataka, India (Hybrid) · Full Time
Be the first to apply
- Experience
- 6+ yrs
- Salary
- —
- Openings
- 1
- Posted
- 5 hours ago
- Work mode
- Hybrid
- Education
- Any graduate
- Eligibility
- Applicants should have completed any graduate degree and meet the experience criteria. Only candidates available to work from Whitefield office three days per week and attend a Saturday interview will be considered.
- Resume
- Required to apply
Where you'll work
Sign in to tell us what does and doesn't work for you here — it sharpens every match we show you.
Job description
About the Company
Quess Corp Limited stands as India's premier staffing and workforce solutions provider, enabling organizations to build future-ready teams through technology-enabled staffing, recruitment, and managed outsourcing. Founded in 2007 and headquartered in Bengaluru, it spans IT, BFSI, telecom, manufacturing, retail, healthcare, and Global Capability Centers, with over 480,000 employees across 8 countries serving more than 3,300 clients worldwide. Quess combines domain expertise with AI-driven digital platforms to deliver innovative workforce solutions, driving employment creation, productivity, and business growth through operational excellence and client-focused services.
Role Responsibilities
- Architect, develop, and maintain scalable data engineering solutions leveraging Apache Spark and Scala.
- Create and manage complex data pipelines using Airflow and other orchestration technologies.
- Utilize Hive, Kubernetes, Docker, and GitHub Actions extensively in development workflows.
- Develop and fine-tune data solutions employing Apache Iceberg.
- Design and deploy data engineering projects on Azure Databricks platforms.
- Work with ADLS, ADF, Delta Lake, Databricks Jobs & Pipelines, Lakeflow Connect, and Unity Catalog for effective data management.
- Operate within both Classic and Serverless Databricks Compute environments.
- Ensure data pipelines are scalable, reliable, and high-performing.
- Adhere to CI/CD and DevOps methodologies using GitHub Actions.
- Investigate and implement AI-assisted development techniques within the software development lifecycle and data engineering processes.
- Collaborate across teams to deliver innovative, business-aligned data solutions.
Candidate Profile and Requirements
- Minimum 6 years of professional experience in Data Engineering.
- Extensive hands-on expertise in Spark and Scala programming.
- Proficient understanding of Azure Databricks ecosystem.
- Strong knowledge of data pipelines, orchestration, and distributed data processing systems.
- Experience with containerization technologies and DevOps tools.
- Ability to work onsite at the Whitefield office in Bangalore for at least 3 days per week.
- Availability for an interview on Saturday is necessary.
Technical Skills
- Apache Spark, Scala
- Airflow orchestration
- Hive data warehousing
- Azure Databricks platform
- Azure Data Lake Storage (ADLS), Azure Data Factory (ADF)
- Delta Lake technology
- Kubernetes and Docker containerization
- GitHub Actions for CI/CD
- Apache Iceberg data format
- Databricks Jobs and Pipelines management
- Lakeflow Connect, Unity Catalog
- Experience with Classic and Serverless Databricks Compute
- Implementation of AI-assisted SDLC and Data Engineering methods
Level
Senior
Minimum education
Bachelor's Degree