Senior Data Engineer

Overview

We are seeking an experienced Senior Data Engineer to design and implement advanced data infrastructure on Google Cloud Platform (GCP). The role centers on developing scalable batch, near-real-time, and streaming data pipelines, along with building analytics-ready data products through efficient ETL/ELT processes.

Key Responsibilities

• Architect and develop scalable data pipelines for batch, streaming, and near-real-time data processing utilizing GCP technologies.
• Create and maintain ETL/ELT data workflows that support diverse analytics applications.
• Work extensively with GCP services including BigQuery, Dataflow, Dataproc, Pub/Sub, Cloud Composer, and Cloud Storage.
• Design and implement modern architectural solutions such as Data Lakes, Lakehouses, Data Warehouses, and Data Meshes.
• Develop robust data and dimensional models, including necessary transformations and SQL optimization.
• Implement comprehensive data quality management practices including validation, reconciliation, observability, metadata management, and data lineage tracking.
• Ensure all data solutions comply with strict security, privacy, governance, and performance standards.
• Setup and manage continuous integration and continuous deployment (CI/CD) pipelines, automated testing, monitoring, and production support frameworks.
• Optimize data pipelines for enhanced performance, scalability, durability, and cloud cost-effectiveness.
• Utilize authorized AI-assisted engineering tools to streamline development, testing, documentation, SQL tuning, and troubleshooting tasks.
• Provide technical leadership; mentor junior engineers and actively participate in architectural and design reviews.

Required Qualifications

• Minimum of 8 years of professional experience in Data Engineering or Data Architecture roles.
• In-depth expertise with Google Cloud Platform (GCP) services.
• Proficient in Python programming and SQL queries.
• Demonstrable hands-on experience building robust ETL/ELT processes and data pipelines.
• Solid experience with BigQuery, Dataflow, Dataproc, Pub/Sub, and Cloud Composer platforms.
• Strong background in data modeling, including dimensional modeling techniques.
• Thorough understanding of data governance principles, data quality assurance, lineage, and observability tools.
• Familiarity with software development tools such as Git, CI/CD frameworks, and automated testing suites.
• Comprehensive understanding of modern cloud data architecture principles and best practices.