Software Development Engineer III

Must-Have Skills:

• Hands-on experience with Databricks and Apache Spark for building and validating scalable data pipelines
• Strong expertise in AWS services including S3, Glue, Athena, Redshift, and Lake Formation
• Proficient in Python, PySpark, and SQL for developing test automation and validation logic
• Experience validating data from various file formats such as JSON, CSV, Parquet, and Avro
• In-depth understanding of data integration workflows including batch and real-time (streaming) pipelines
• Strong ability to define and automate data quality checks: schema validation, null checks, duplicates, thresholds, and transformation validation
• Experience designing modular, reusable automation frameworks for large-scale data validation
• Skilled in integrating tests with CI/CD tools like GitHub Actions, Jenkins, or Azure DevOps
• Familiarity with orchestration tools such as Apache Airflow, Databricks Jobs, or AWS Step Functions
• Hands-on experience with API testing using Postman, pytest, or custom automation scripts
• Proven track record of leading and mentoring QA/test engineering teams
• Ability to define and own test automation strategy and roadmap for data platforms
• Strong collaboration skills to work with engineering, product, and data teams
• Excellent communication skills for presenting test results, quality metrics, and project health to leadership
• Contributions to internal quality dashboards or data observability systems
• Awareness of metadata-driven testing approaches and lineage-based validations
• Experience working with agile Testing methodologies such as Scaled Agile.
• Familiarity with automated testing frameworks like Selenium, JUnit, TestNG, or PyTest.