Senior Data Engineer - Project Delivery

Haystack
London, GB
On-site

Job Description

Senior Data Engineer

We're working with a leading project delivery consultancy driving innovation across critical UK industries on this exciting opportunity.

Are you ready to build the future of cloud-first data systems? We are looking for a Senior Data Engineer to design and manage scalable pipelines using Azure Data Factory, Synapse, and Databricks to power high-impact analytics and AI. This is a chance to join a people-first organization during a massive growth phase where you can influence the roadmap of modern data platforms.

The Role

  • Design, build, and optimize complex ETL/ELT workflows using Azure Data Factory, Synapse Analytics, and Databricks to ensure high-quality data products.
  • Architect and manage advanced data lakes and warehouses utilizing Delta Lake, Parquet, and Azure Data Lake Storage Gen2.
  • Collaborate directly with AI Engineers and Data Analysts to deliver robust data assets for automation and machine learning workstreams.
  • Champion modern DataOps practices, including CI/CD via Azure DevOps and Infrastructure as Code with Terraform or Bicep.
  • Provide technical leadership and mentorship to junior engineers while shaping the evolution of the organization’s data platform (including Microsoft Fabric).

What You'll Need

  • Proven professional experience in Data Engineering or Cloud Development with a deep mastery of the Microsoft Azure ecosystem.
  • Strong technical proficiency in Python (PySpark, Pandas) and SQL Server for complex data manipulation and pipeline automation.
  • Hands-on experience with Delta Live Tables, REST APIs, and GitHub Actions for streamlined deployment and monitoring.
  • Solid understanding of modern data architecture, security best practices, and the ability to lead technical workstreams independently.
  • An ambitious mindset with interest in emerging tech like Microsoft Fabric, real-time streaming, and AI-enhanced ETL.

What's On Offer

  • Competitive salary of £60,000 - £80,000 in a thriving, scaling business.
  • Genuine people-first culture with significant investment in professional development and mentorship.
  • Opportunity to work on critical infrastructure projects that impact industries UK-wide.
  • Collaborative hybrid working environment in the heart of London.

Apply via Haystack today!

Skills & Requirements

Technical Skills

PythonPySparkPandasSQL ServerAzure Data FactorySynapse AnalyticsDatabricksDelta Live TablesREST APIsGitHub ActionsTerraformBicepleadershipcommunicationcloud developmentdata engineeringdata architecturesecurity best practicesAI-enhanced ETL

Salary

£60,000 - £80,000

year

Level

senior

Posted

3/28/2026

Continue to LinkedIn

You will be redirected to the job posting on LinkedIn.