Senior Quality Engineer – Data Pipelines (AWS) Introduction:
We are seeking a Senior Quality Engineer to own test planning and validation of data pipelines within an AWS-based data lakehouse environment. This role will work closely with developers, DevOps, and data teams to ensure data integrity and reliability across CI/CD pipelines and cloud workflows. The position involves designing and maintaining test automation, independently analyzing logs to troubleshoot failures, and contributing to stable and reliable data processing using AWS services.
Responsibilities:
- Own end-to-end test planning and execution for AWS-based data lakehouse pipelines and integrations
- Design, develop, and maintain automated test scripts to validate data pipelines and support CI/CD workflows
- Validate data workflows using AWS services including S3, Glue, and Athena
- Independently support and troubleshoot CI/CD pipelines using CodeCommit, CodeBuild, and CodePipeline
- Review build and deployment logs to identify failures and perform root cause analysis
- Work with Terraform/Terrahub components within build pipelines (CodeBuild)
- Perform functional, regression, and data validation testing across releases
- Track, document, and drive defects through resolution in collaboration with development teams
- Define test strategies, scope, and estimates for releases and test cycles
- Collaborate with cross-functional teams to support UAT and production readiness
- Communicate testing progress, risks, and results to stakeholders
Requirements:
Required Skills:
- 14+ years overall experience in software quality, testing, and data pipeline validation
- 5+ years in testing AWS-based data pipelines and cloud workflows
- Experience working in Agile environments (Scrum or Kanban)
- Familiarity with cloud-based and DevOps-driven development environments
- Working knowledge of SQL and relational databases
- Experience with software testing tools (e.g., Jira, qTest, ALM, Selenium)
- Basic proficiency with Git version control (clone, commit, push, pull, branching)
- Basic command-line proficiency (Linux, PowerShell, CMD)
- Hands-on experience with AWS services: S3, Glue, Athena, CodeCommit, CodeBuild, CodePipeline
- Experience with Terraform/Terrahub in CI/CD pipelines
- Log analysis and troubleshooting in build/deployment pipelines
Preferred Skills:
- Bachelor’s degree in Computer Science, Engineering, Information Systems, or equivalent experience