Senior Staff Applied AI Engineer

OpenLoop Health

Toronto

Remote

Who this role is best for

Best suited to technical leaders with hands-on AI expertise working in healthcare or regulated industries.

Best fit for

Candidates with 12+ years of software engineering and technical leadership experience in AI systems
— “12 or more years of software engineering experience, with real technical leadership experience”
Individuals with production experience in LLM-based systems and a focus on evaluation and observability
— “Build the evaluation systems that prove our AI is actually good, not just 'seems to work'”
Candidates who have led technical direction for small teams from scratch or near scratch
— “Experience leading a small team's technical direction from zero, or close to it”
Professionals with a strong background in AI infrastructure and tools development
— “This pod will eventually split into two teams, AI Infrastructure and AI Tools”

Things to consider

The role demands a high level of responsibility in shaping long-range AI strategy for the company
— “you'll also carry the long-range technical direction for AI across the whole company”
Candidates must be prepared to work closely with business and clinical stakeholders on AI trade-offs
— “Explain AI trade-offs in plain language to product, operations, and clinical stakeholders”

How to stand out

Highlight experience in building evaluation systems for AI in production environments
— “Build the evaluation systems that prove our AI is actually good”
Emphasize your ability to mentor engineers and raise technical standards through reviews
— “Raise the bar through code and design review, and mentor engineers who are new to AI”
Showcase your track record in managing AI operations and model access in multi-provider scenarios
— “Make the call on model access and operations: routing across providers”
Demonstrate your capability to build internal platforms used by other engineering teams
— “Experience building an internal platform used by other engineering teams”
Demonstrate your expertise in securing patient data within AI systems
— “Keep patient data protected in every AI system you touch”

Pace · Fast PacedCollaboration · HighAutonomy · MediumDecision Impact · Team

Derived from job-description analysis by Serendipath's career intelligence engine.

What success looks like

AI agent workflows
secure AI systems

Typical background

12+ years in software engineeringexperience in AI engineering

Skills & requirements

Required

AI EngineeringAgent RuntimeEvaluation SystemsModel Access And OperationsObservability

Preferred

Healthcare ExperienceGCP ExperienceInternal Platform Building

Stack & domain

AI EngineeringLlmsAgent RuntimeEvaluation SystemsCloud InfrastructureData ProtectionLeadershipCommunicationProblem-solvingTeamworkMentorshipHealthcareTelehealth

About the role

Original posting from OpenLoop Health via Ashby

ABOUT OPENLOOP
OpenLoop was co-founded by CEO, Dr. Jon Lensing, and COO, Christian Williams, with the vision to bring care anywhere. Our telehealth support solutions are thoughtfully designed to streamline and simplify go-to-market care delivery for companies offering meaningful virtual support to patients across an expansive array of specialties, in all 50 states.
ABOUT THE ROLE
OpenLoop is building the future of healthcare infrastructure. We are hiring a Senior Staff Applied AI Engineer to be one of the first technical leader for AI at OpenLoop .
As one of the earliest hires, the decisions you make become the foundation everyone else builds on: how our AI agents run, how we prove they are good, how we keep them safe and affordable, and how other teams build agents on our platform.
WHAT YOU'LL DO
Agent Runtime & Orchestration: Set the technical direction for how agents run, call tools, hold context and hand off work, including failure handling, retries, timeouts and sandboxing. Decide when to build and when to adopt frameworks.
Evaluation: Build the systems that prove our AI is actually good, including regression tests and AI-graded evals, and help each team define what "good" means for their agents.
Model Access & Operations: Set up access to models from multiple providers, route between them, manage version upgrades and plan fallbacks for when a provider fails.
Observability & Cost: Trace what agents did and why, monitor behavior in production, and attribute AI spend to each agent and use case with budgets and guardrails.
Grounding & Retrieval: Connect agents to real, messy company data and documents through retrieval (RAG), context design and prompt engineering, so answers are based on facts.
Workflow Design: Partner with operations teams to turn human workflows, such as patient support, into agent workflows, and decide which steps should stay with a person.
Security, Privacy & Safety: Keep patient data (PHI) protected in every AI system, control what agents can access, and defend against misuse such as prompt injection.
Cloud & Data Foundations: Make sound infrastructure decisions on GCP and partner with the Data Platform teams who provide the governed data agents use.
Technical Leadership: Set and document direction when the right answer is unclear, make build-versus-buy calls and revisit them as tools change, keep the platform general enough for many teams, and mentor engineers who are new to AI through code and design review.
Cross-functional Work: Explain AI trade-offs in plain language to product, operations and clinical stakeholders, including when the honest answer is "the AI isn't ready for this yet."
Other duties as assigned.
WHO YOU ARE
REQUIRED QUALIFICATIONS:
Bachelor's degree in Computer Science, Engineering, or a related technical field
Proven Technical Leadership: 12+ years of software engineering experience, with a track record at tech lead, staff or principal-level scope.
Production LLM Experience: 2+ years building and running LLM-based systems in production that real users relied on, not only demos, prototypes or personal projects.
Core AI Depth: Deep, hands-on expertise in agent runtime or evaluation, a second area of real depth, and working knowledge across the rest of the AI engineering stack.
Evaluation Rigor: A track record of measuring AI quality with real evaluation sets and test harnesses, and reporting results with numbers.
Zero-to-One Leadership: Experience leading a small team's technical direction from zero, or close to it, in a greenfield setting.
Excellent communication skills with the ability to explain complex technical concepts and trade-offs to non-technical stakeholders
PREFERRED QUALIFICATIONS:
Regulated-Industry Experience: Healthcare (PHI, HIPAA), finance or insurance.
GCP Experience: Running production services on Google Cloud.
Internal Platforms: Built a platform used by other engineering teams.
Classic ML Background: Machine learning experience beyond LLMs.
Team Scaling: Helped grow a single pod into multiple teams.
OUR BENEFITS
In addition, for salaried positions you would also be eligible for:
Medical, Dental, and Vision plans
Flexible Spending/Health Savings Accounts
Flexible PTO
401(k) + Company Match
Life Insurance, Pet insurance, and more
OUR COMPANY
We have a relatively flat organizational structure here at OpenLoop. Everyone is encouraged to bring ideas to the table and make things happen. This fits in well with our core values of Autonomy, Competence and Belonging, as we want everyone to feel empowered and supported to do their best work.
Sound like a good fit? We’d love to meet you.

Source: OpenLoop Health careers (Ashby)