Databricks Developer

Role Overview

Join an IT service organization dedicated to driving digital transformation through advanced technological solutions and deep industry insight. As a Databricks Developer based in Hyderabad, you will design, build, and maintain complex data processing systems leveraging Azure Databricks and related technologies.

Primary Responsibilities

• Create and manage efficient data pipelines utilizing Azure Databricks.
• Handle large-scale datasets using PySpark and Spark SQL processing.
• Design and implement ETL and ELT workflows sourcing from diverse data inputs.
• Enhance data quality while optimizing system performance and operational costs.
• Oversee Delta Lake management, Databricks notebooks, and orchestrate Databricks workflows.
• Collaborate effectively with data engineering and business units to deliver data solutions.
• Integrate additional Azure components like Azure Data Lake Storage (ADLS) and Azure Data Factory (ADF) in data operations.

Candidate Profile

• Demonstrated proficiency in Azure Databricks platform and core Databricks functionalities.
• Advanced skills with PySpark, Spark SQL, and distributed computing methods.
• Strong familiarity with Delta Lake and modern Lakehouse architectural paradigms.
• Experience with Azure Data Factory, ADLS, and Azure DevOps tools is advantageous.
• Solid knowledge of ETL and ELT processes along with data modeling techniques.
• Competent in troubleshooting and boosting system performance.
• Excellent analytical thinking and adept problem-solving skills.

Eligibility

Applicants must hold at least a graduate degree to qualify for this position.

Compensation

The salary range offered for this role is between 1,000,000 and 1,400,000 INR annually.