- Experience
- 5+ yrs
- Salary
- INR 1,400,000 – INR 2,000,000 / year
- Openings
- 1
- Posted
- 3 ದಿನಗಳು ಹಿಂದೆ
- Work mode
- In office
- Eligibility
- Open to all applicants; no graduation requirement specified.
- Resume
- Required to apply
Where you'll work
Sign in to tell us what does and doesn't work for you here — it sharpens every match we show you.
Job description
Job Overview
This role focuses on architecting and strategizing automated testing for data pipelines including batch and streaming processes. It involves ensuring data quality and observability through sophisticated testing frameworks and integration with data operations practices.
Key Responsibilities
- Design and implement test automation architecture for data pipelines and backend APIs.
- Develop data testing frameworks using SQL-based assertions and schema validations.
- Conduct proactive destructive testing to handle schema changes, late-arriving data, and partial failures.
- Perform extensive ETL and streaming validations at scale, including incremental and backfill data checks.
- Integrate data quality and observability tools such as Great Expectations and Soda to monitor accuracy, completeness, uniqueness, and timeliness.
- Enforce CI/CD and DataOps by creating pre-merge gates and release blockers with selective and parallel test executions.
- Manage test data through synthetic data generation, masking, and anonymization techniques.
- Conduct performance and reliability tests focusing on pipeline and query benchmarks, concurrency, and stress testing.
- Validate security and compliance aspects including PII/PHI exposure, encryption, and access control.
- Collaborate cross-functionally with data engineers, platform teams, and architects to embed quality early and address design issues affecting reliability.
- Analyze production incidents and failure patterns to minimize flaky tests and false alarms.
Skills and Experience Required
- Strong programming capabilities and framework development experience.
- Proficiency in the data engineering ecosystem and streaming systems.
- Experience with data quality monitoring and observability tools.
- Knowledge of CI/CD and DevOps practices within data operations.
- Familiarity with cloud infrastructure and related technologies.
- Optional but preferred skills include API contract testing (e.g., PACT), basic UI automation, and exposure to data mesh or data product architectures.
- Must have over 5 years of experience in automation or quality engineering with at least 3 years dedicated to building frameworks for scalable data platforms.