Loading description…
Vacancy description
r3 consultant
Jaipur, भारत
Job Title: Senior QA Data Engineer
Location: Remote
Budget: 18- 25 LPA
No of positions: 1
Shift: 11:00 AM to 8:00 PM IST
Experience: 5+ Years
Position Overview :
We are looking for an experienced Sr. QA Data Engineer to join our Data Engineering team and take ownership of the quality, accuracy, consistency, and reliability of our data pipelines and data-driven systems. The ideal candidate will have strong hands-on experience in Data Testing, ETL/ELT Testing, Database Testing, SQL, Python, Apache Spark, and Databricks. The candidate should be comfortable validating complex data transformations, investigating data discrepancies, and testing end-to-end data workflows across multiple sources, processing layers, and downstream systems. You will work closely with Data Engineers, Software Engineers, Database Architects, Data Analysts, Data Scientists, and Product teams to ensure data is processed correctly and delivered reliably to downstream consumers. This role requires strong analytical and problem-solving skills, attention to detail, and the ability to
independently understand complex data flows, develop comprehensive test strategies, automate data validation, investigate defects, and drive quality across the data engineering ecosystem.
What You'll Be Doing
Design and execute functional, integration, regression, and end-to-end testing for data pipelines and data platforms.
Validate ETL/ELT workflows, data transformations, business rules, and source-to-target data accuracy.
Test Apache Spark/PySpark, Databricks, and Delta Lake data processing workflows.
Perform advanced SQL-based data validation, reconciliation, and data-quality testing.
Validate data for accuracy, completeness, consistency, uniqueness, integrity, and schema compliance.
Test APIs and integrations across data sources, databases, pipelines, and downstream systems.
Develop and maintain Python/SQL-based test automation for data validation and regression testing.
Design and execute positive, negative, edge-case, failure, retry, and recovery scenarios across data workflows.
Investigate data discrepancies and work with Data Engineers, Developers, Analysts, and Data Scientists to identify root causes and resolve issues.
Review requirements, technical designs, and data flows to identify QA risks, dependencies, and coverage gaps early in the development lifecycle.
Maintain comprehensive test cases, automation suites, defect documentation, test evidence, and release validation for assigned data products.
Drive continuous improvement of QA processes, data validation frameworks, automation coverage, and overall data quality.
Qualifications
Bachelor's degree in Computer Science, Engineering, Information Technology, or a related field.
5+ years of relevant QA/testing experience, preferably working with Data Engineering, Big Data, or data-intensive applications.
Strong hands-on experience in ETL/ELT testing, database testing, and data validation.
Advanced SQL skills, including complex joins, subqueries, CTEs, window functions, aggregations, and data reconciliation.
Practical experience testing Apache Spark/PySpark data processing workflows.
Hands-on experience with Databricks and Delta Lake.
Strong understanding of Big Data concepts, distributed data processing, data pipelines, and data architecture.
Experience validating Parquet, JSON, CSV, and other structured and semi-structured data formats.
Strong programming/scripting skills in Python for test automation and data validation.
Experience with API and integration testing.
Working knowledge of Linux/Unix environments and shell scripting.
Understanding of Data Lakes, Lakehouse architecture, Data Warehousing, and ETL/ELT pipelines.
Experience with Git, defect management tools, and CI/CD-based testing workflows.
Experience working in an Agile/Scrum environment.
Strong analytical, troubleshooting, and root-cause analysis skills.
Ability to work independently, take ownership, and collaborate effectively with cross-functional teams.
Excellent written and verbal communication skills in English.