Senior Lead Engineer - ETL Testing

Job Summary

We are seeking a seasoned QA professional with deep expertise in data analysis, ETL testing, data warehousing, database validation, and BI testing. The ideal candidate will bring practical experience with Python-based data testing automation, SQL query validation, Azure Data Factory pipelines, Snowflake, and Power BI. This role focuses on end-to-end data validation, test automation, defect lifecycle management, and close collaboration with cross-functional teams to ensure data accuracy, completeness, and overall quality.

Key Responsibilities

• Perform end-to-end validation across ETL processes, data pipelines, data warehouses, and database systems.
• Verify data accuracy, completeness, consistency, and integrity across multiple platforms.
• Test and validate file ingestion workflows, including CSV, TXT, and Parquet files stored in Azure Data Lake Storage (ADLS).
• Execute and validate SQL queries in Snowflake, Azure SQL Managed Instance, and Amazon Redshift.
• Execute, monitor, and validate Azure Data Factory (ADF) pipelines, including data movement across environments.
• Conduct Power BI report testing and perform data validation.
• Design and execute testing strategies covering data validation, UI testing, and REST API testing.
• Create and maintain Python-based test automation scripts for database validation, file handling, API testing, and Azure integration.
• Prepare test plans, test scenarios, test cases, test data, and maintain test documentation.
• Manage the full defect lifecycle—identification, logging, tracking, reporting, and resolution.
• Track and report QA metrics and overall quality status to relevant stakeholders.
• Support Agile estimation activities, including T-shirt sizing and Planning Poker.
• Provide technical direction to team members and collaborate with developers, QA teams, and business stakeholders.

Mandatory Technical Skills

Data & ETL Testing

• ETL Testing and Data Pipeline Validation
• Data Warehousing and Database Testing
• Data Quality, Data Integrity, and Data Reconciliation
• File Ingestion and Validation
• CSV, TXT, and Parquet File Formats
• Power BI Testing and Validation

SQL & Database Technologies

• Advanced SQL Query Writing and Optimization
• Snowflake
• Azure SQL Managed Instance
• Amazon Redshift
• Database Validation Across Multiple Platforms

Python & Test Automation

• Python-Based Data Testing Automation
• Python Libraries: pandas, pyodbc, requests, and SQLAlchemy
• Database Cursors and Database Connectivity
• File Handling and Data Comparison
• REST API Testing
• Azure SDKs

Azure & DevOps

• Microsoft Azure Cloud Services
• Azure Data Factory (ADF)
• Azure Data Lake Storage (ADLS)
• Azure DevOps Pipelines
• Azure DevOps Test Plans
• Pipeline Execution and Validation

QA & Testing Practices

• Test Planning and Test Case Design
• UI Testing and REST API Testing
• Defect Management and Reporting
• Regression Testing
• QA Metrics and Test Documentation
• Agile/Scrum Methodologies

Good-to-Have Skills

• Pytest and Robot Framework
• Data Testing Automation Framework Development
• Automated Regression Test Suites
• Data Testing Across On-Premises and Azure Cloud Environments
• Advanced Snowflake Testing
• Azure DevOps Test Plans and Pipeline Automation
• Experience with reusable and scalable test automation frameworks

Candidate Requirements

• 10+ years of experience in QA, data analysis, ETL, data warehouse, BI, and data quality testing.
• Strong hands-on experience in Python, SQL, and data validation across multiple databases.
• Experience testing Azure Data Factory pipelines and validating files in ADLS.
• Strong understanding of Power BI testing, UI testing, and REST API testing.
• Experience designing test strategies, preparing test data, and managing defects.
• Strong analytical, troubleshooting, communication, and stakeholder collaboration skills.
• Ability to guide team members and work effectively in Agile delivery environments.