Databricks Developer

About 2P Perfect Presentation

Since 2004, 2P Perfect Presentation has played a pivotal role in Saudi Arabia's ICT sector by delivering comprehensive technology solutions that facilitate digital transformation and boost operational effectiveness. Renowned for its innovative approach and strong service focus, the company offers end-to-end solutions through specialized business segments backed by expert market knowledge. It is committed to connecting with motivated professionals eager to contribute and thrive in a progressive work environment.

Role Overview and Responsibilities

The organization is looking for a skilled Databricks Developer to design, develop, and manage data platforms that support both enterprise and government projects. The successful candidate will be responsible for constructing scalable data pipelines in Databricks and contributing to front-end applications using React, which display data via dashboards, reports, and data management tools.

• Create, develop, and sustain ETL and ELT pipelines in Databricks utilizing PySpark, Spark SQL, and Delta Lake technologies.
• Apply medallion architecture principles (bronze, silver, gold layers) to handle batch and streaming data processes.
• Build and maintain Databricks notebooks, schedule jobs, and oversee workflows efficiently.
• Set up and regulate clusters, manage job scheduling, and optimize workload execution.
• Integrate diverse data sources, including SQL Server databases, API endpoints, file streams, and other external enterprise systems.
• Enhance Spark job performance and scalability while keeping operational costs in check.
• Implement rigorous data validation, continuous pipeline monitoring, logging, and error handling routines.
• Develop and maintain React components to visualize data outputs effectively for end-users.
• Seamlessly integrate React-based interfaces with REST APIs and supporting backend services.
• Collaborate closely with system architects, DevOps professionals, and QA teams through all phases of design, code validation, testing, deployment, and live support.
• Investigate and resolve issues within data pipelines and user-facing applications by conducting root cause analyses.
• Adhere strictly to data security protocols, governance standards, and regulatory compliance guidelines.

Qualifications and Experience

• Bachelor’s degree in Computer Science, Software Engineering, Information Systems, or an equivalent field; or comparable practical experience.
• A minimum of four years in hands-on Databricks development within live production settings.
• Proven expertise in data engineering and delivery of robust ETL/data pipeline solutions.
• Proficiency with PySpark and/or Scala programming languages, complemented by strong SQL capability.
• Practical experience managing Delta Lake environments and orchestrating Databricks workflows.
• Deep knowledge of dimensional modeling techniques, including star schemas and Slowly Changing Dimensions.
• Experience in React development encompassing functional components, hooks, state handling, and API integrations.
• Familiarity with version control using Git and continuous integration/continuous deployment (CI/CD) pipelines.
• Excellent analytical, problem-solving, teamwork, and communication skills.
• Preference will be given to candidates who are ready to join immediately.

Preferred Expertise

• Hands-on experience with cloud platforms such as Azure Data Factory, Azure Data Lake Storage, Azure Synapse Analytics, Azure Databricks or analogous AWS/Google Cloud services.
• Proficiency in integrating data among multiple enterprise-level systems.
• Knowledge of Azure DevOps platform.
• Experience with streaming technologies like Structured Streaming, Kafka, or Event Hubs.
• Familiarity with Kubernetes and container orchestration environments.
• Understanding of enterprise or government data security and compliance mandates.
• Possession of Databricks Data Engineer Associate or Professional certification.
• TypeScript programming experience alongside React development.

Skills Summary

Databricks, Data Engineering, PySpark, Scala, Spark SQL, Delta Lake, ETL/ELT Pipeline Design, Medallion Architecture Implementation, Databricks Workflow Management, SQL Server Integration, Dimensional Data Modeling, React Front-end Development, TypeScript, REST API Integration, Git Version Control, CI/CD Pipeline Management, Azure Cloud Platform, Streaming Data Technologies, Data Security Compliance.