DataOps Engineer (AWS / Databricks)

ITTConnect is seeking a DataOps Engineer expert in AWS and Databricks for a direct-hire full time position with a client that is a large financial institution.

Position is hybrid

Key Responsibilities:

• Lead the design and implementation of the cloud data infrastructure, with a focus on AWS + Databricks.
• Define and implement the DataOps strategy for a cloud native data platform.
• Design, implement, and maintain CI/CD pipelines for data and infrastructure, using Terraform and automation best practices.
• Define implementation standards for PySpark based data pipelines, Repository structure and organization, Deployment, versioning, and rollback strategies
• Implement data governance policies, including cataloging, lineage, ownership, and data layers.
• Ensure data security controls, including Access management, encryption and environment segregation
• Define and operationalize data quality practices, such as validations, SLAs, monitoring, and observability.
• Work closely with engineering teams to standardize development workflows.
• Support the migration of data from the on premise environment to the cloud, ensuring reliability and consistency.
• Act as a technical authority for architectural decisions and the evolution of the data stack.

Requirements

• 10+ years of experience in IT.
• MUST HAVE experience with AWS (S3, IAM, VPC, monitoring and security services), Databricks and PySpark.
• Highly desirable previous experience working for Financial Services / Banks.
• Solid experience in DataOps, Data Engineering, or Platform Engineering.
• Hands on experience building cloud based data platforms.
• Experience with Infrastructure as Code (Terraform).
• Experience with data pipelines and orchestration concepts.
• Proven experience with CI/CD for data platforms, data governance, data quality, data security
• Ability to define technical standards and lead end to end implementations.

Nice to have:

• Experience migrating platforms from on premise to cloud.
• Knowledge of Lakehouse architectures.
• Experience in regulated environments (financial services, insurance, healthcare).
• Experience with data observability tools.
• AWS, Databricks, or Data Engineering certifications.
• Bachelor’s degree in Computer Science or a related field