Data Engineer – Data Quality & Governance
Location: Abu Dhabi
Reports To: Data & AI Lead
Role Overview
We are seeking an experienced Data Engineer to join our Data & AI team, delivering enterprise-grade data platform capabilities. This role focuses on building Python-based ETL pipelines across a Medallion architecture (Bronze / Silver / Gold) and developing data quality rules in Python, while collaborating closely with platform specialists using the Informatica governance stack (IDQ, EDC, Axon).
The ideal candidate brings strong Python engineering skills with a conceptual understanding of Informatica’s on-premises data quality and governance tools. Prior hands-on Informatica experience is welcome but not mandatory—structured ramp-up support will be provided.
Key Responsibilities
- Design, build, and maintain Python-based ETL pipelines across Bronze, Silver, and Gold layers.
- Develop and operationalise data quality rules in Python (validity, completeness, consistency, uniqueness, accuracy, timeliness).
- Contribute to Informatica IDQ 10.5 activities (mapplets, profiling, scorecards) alongside platform specialists.
- Support data cataloguing and lineage activities using Informatica EDC.
- Assist governance workflows in Informatica Axon, including business glossary and DQ score visibility.
- Build cross-system DQ checks (referential integrity, reconciliation, deduplication).
- Integrate DQ outputs into Power BI dashboards.
- Maintain a modular, testable Python DQ framework with unit test coverage.
- Support Talend-based ingestion at the Bronze layer.
- Participate in root-cause analysis of data quality issues.
Required Skills & Experience
- Strong Python development skills (pandas, PySpark).
- Experience with production-grade ETL pipelines, logging, and error handling.
- Understanding of Medallion architecture and data lake design.
- Working knowledge of SQL and relational databases.
- Familiarity with CI/CD practices and Git.
- Conceptual understanding of:
- Informatica IDQ (Developer Tool, profiles, scorecards)
- Informatica EDC (catalog, lineage, classification)
- Informatica Axon (governance, glossary, stewardship workflows)
Desirable Skills
- Experience with Talend ingestion workflows
- Exposure to AI/ML for Data Quality (anomaly detection, entity resolution)
- Power BI dashboard development
- Informatica IDQ–Axon integration awareness
- Knowledge of data governance frameworks and regulatory reporting
Additional Information
- Candidates must be comfortable working with sensitive data and governance controls.
- Experience in Agile delivery teams is preferred.
- Strong documentation and stakeholder communication skills are essential.
Work Location: In person