Lead Data Engineer

About Ascendion

Ascendion Engineering Private Limited is a technology-driven firm based in India that delivers engineering and professional services to support diverse business needs. With a focus on AI-powered software solutions, Ascendion collaborates with Global 2000 companies worldwide to tackle complex challenges using data, product engineering, experience design, and workforce transformation. Their unique Engineering to the Power of AI™ approach combines expert engineers and AI agents to produce impactful, trusted, and scalable results. The company fosters an environment promoting learning, collaboration, career growth, inclusion, and partnership.

Role Overview

The Lead Data Engineer role centers on architecting, implementing, and enhancing data pipelines and platforms that empower advanced analytics and generate business intelligence. This position demands a proficient background in Databricks, SQL, cloud platforms such as AWS, Azure, or GCP, and workflow tools like Airflow. The engineer will ensure efficient, secure, and scalable data architectures while working closely with cross-disciplinary teams.

Key Responsibilities

• Create and sustain ETL pipelines and workflows utilizing Databricks technologies and Airflow orchestration.
• Possess in-depth knowledge of Databricks Delta Live Tables and Unity Catalog for data management.
• Develop reusable frameworks and templates to support ETL/ELT processes specifically for telemetry data ingestion.
• Design cloud-native solutions in compliance with AWS best practices, emphasizing security and scalability.
• Maintain high standards of data quality, governance, and integrity throughout data systems.
• Enhance and optimize SQL query performance and data model scalability.
• Implement comprehensive monitoring and observability frameworks for data workflows to ensure reliability.
• Engage closely with architects, leads, analytics, and business teams to translate data requirements into robust technical implementations and actionable BI insights.
• Experience or interest in Agentic AI methodologies to automate data quality checks and metadata management is a plus.

Required Skills & Experience

• Minimum of eight years in data engineering roles, including at least three years focused on Databricks technologies.
• Expertise in Databricks components such as PySpark, Spark SQL, Delta Lake, Unity Catalog, and Delta Live Tables.
• Strong proficiency in SQL for data modeling and query tuning.
• Practical experience with AWS services including S3, Glue, Kinesis, Redshift, API Gateway, SNS, and Lambda.
• Familiarity with Airflow for orchestrating complex workflows.
• Additional skills advantageous: DBT, Splunk, Atlan, Java programming, API development, and Power BI for visualization.
• Excellent analytical, problem-solving abilities with high attention to detail.

Location

This role is based in Bengaluru, India, requiring onsite presence.