Role: Mid-Level QA / Testing Automation Engineer
Experience Requirements:
- Total QA/Testing Experience: 5+ years
- Data Testing Experience: 3+ years in Big Data, Hadoop, or Cloud Data Warehouse environments
- Good to have: 1+ year of Databricks experience (testing pipelines)
- Strong automation experience transitioning from manual SQL checks to Python-based testing frameworks
- Experience in migration automation testing using Python
Required Certifications:
- Good to have: Databricks Certified Data Engineer Associate
- Preferred: ISTQB Foundation or Advanced Level (Test Automation Engineer)
Core Technical Skills:
- Data validation using Great Expectations / Pandera
- Strong understanding of Delta Live Tables (DLT) expectations
- Advanced SQL for large-scale data reconciliation
- Python (PySpark), pytest-spark, and automated notebook testing
- Data reconciliation and source-to-target validation
- Performance, scalability, and end-to-end pipeline testing
- Schema evolution and integration testing
- Governance & security testing (Unity Catalog, RLS, masking, lineage)
Preferred Background:
- Data-first mindset with strong focus on data validation
- Experience with Git (branching/merging)
- Basic understanding of distributed systems and Spark
Key Responsibilities:
- Develop and implement test strategies for data platforms
- Automate data validation using Python within CI/CD pipelines
- Perform defect analysis and collaborate with data engineers
- Conduct regression testing to ensure data accuracy and consistency
