Senior AI Quality Engineer: Enhance AI Performance and Standards

Roofr
Toronto, CA; US
Remote

Job Description

Become a cornerstone in AI quality assurance as a Senior AI Quality Engineer. Lead the charge in establishing robust testing frameworks and quality standards while working in a remote-first setting.

This is an exciting opportunity to impact AI features across products. You will be embedded within the AI Platform team, ensuring high-quality standards from the outset of development. Your role includes coaching teams and designing evaluation methodologies for LLM-powered features to assure consistency and safety in performance.

Key Responsibilities:

  • Establish best practices for AI testing
  • Create an LLM evaluation framework
  • Implement QA gates into CI/CD processes
  • Design and manage review workflows for AI outputs
  • Train teams on AI evaluation patterns

Requirements:

  • 5–8 years in quality assurance or software engineering
  • Proven experience in LLM eval frameworks
  • Coding proficiency and automation expertise
  • Knowledge of LLM APIs and their quality challenges
  • Strong collaboration and mentorship abilities

Drive quality innovation and set the gold standard for AI performance in this pivotal role.

#J-18808-Ljbffr

Skills & Requirements

Technical Skills

Quality assuranceSoftware engineeringLlm eval frameworksCoding proficiencyAutomation expertiseLlm apisPlcScadaCollaborationMentorshipAiLlm

Employment Type

FULL TIME

Level

senior

Posted

4/10/2026

Apply Now

You will be redirected to Roofr's application portal.