Clinical AI Evaluation Intern

Qualified Health Pbc
Palo Alto, CA
HybridCareer-pivot friendly

Who this role is best for

Clinical professionals with medical scribe experience and a focus on detail will find this evaluation role in healthcare AI. The hybrid work arrangement and structured, non-patient-facing tasks align with candidates seeking technical and clinical collaboration in a mission-driven environment.

Best fit for

  • Medical scribes or clinical professionals with strong written reasoning skills and interest in AI safety
    — “reviewing, evaluating, and labeling the outputs of our GenAI products
  • Individuals with a background in clinical documentation and a desire to influence AI product development
    — “your input will shape how this function is built as it grows
  • Candidates comfortable with structured, detail-intensive work and eager to learn AI evaluation processes
    — “the work is structured and detail-intensive, and the quality of our product depends on it being done carefully

Things to consider

  • Requires 3 days per week onsite in Palo Alto despite hybrid flexibility
    — “Able to work onsite in Palo Alto 3 days/week
  • Six-month commitment with potential extension based on performance
    — “the role runs for 6 months at 20–40 hours per week, with full time preferred, and may be extended based on performance and team needs

How to stand out

  • Highlight experience with clinical documentation and structured data evaluation in your resume
    — “Working fluency in medical terminology and clinical documentation
  • Emphasize your ability to explain reasoning clearly and consistently in writing
    — “the ability to explain your reasoning clearly in writing
  • Show interest in AI/ML in healthcare and mention any related coursework or projects
    — “Interest in AI/ML in healthcare and in how clinical AI is evaluated for safety and accuracy
Pace · SteadyCollaboration · MediumAutonomy · MediumDecision Impact · Individual

Derived from job-description analysis by Serendipath's career intelligence engine.

What success looks like

  • Accurate labeling
  • Quality assurance
Typical background
Medical scribe experienceClinical education

Skills & requirements

Required

Medical TerminologyClinical DocumentationAttention To Detail

Preferred

EHR SystemsClinical Research

About the role

Original posting from Qualified Health Pbc via Ashby

Transform healthcare with us.

At Qualified Health, we're redefining what's possible with Generative AI in healthcare. Our infrastructure provides the guardrails for safe AI governance, healthcare-specific agent creation, and real-time algorithm monitoring—working alongside leading health systems to drive real change.

This is more than just a job. It's an opportunity to build the future of AI in healthcare, solve complex challenges, and make a lasting impact on patient care. If you're ambitious, innovative, and ready to move fast, we'd love to have you on board.

Join us in shaping the future of healthcare.

Job Summary:

Qualified Health is seeking a Medical Scribe Intern to join the clinical team responsible for reviewing, evaluating, and labeling the outputs of our GenAI products. Working alongside our physicians, you will compare AI-generated clinical content against defined gold standards, and label the data that trains and evaluates our models.

The ideal candidate has worked as a medical scribe, or brings comparable clinical education or professional experience, and is fluent in medical terminology and clinical documentation. This is a paid, hourly internship. The work is structured and detail-intensive, and the quality of our product depends on it being done carefully. You will be trained and mentored directly by our physicians, who bring their clinical perspective to building products. Our physicians will review your work and give you direct feedback on it, and your input will shape how this function is built as it grows.

This role is not patient-facing and does not accrue clinical contact hours. What it offers instead is a close look at how clinical AI is evaluated for safety before it reaches patient care. It is a chance to learn how these tools are tested and improved, and to see firsthand how they are starting to change the way care is delivered.

Key Responsibilities:

  • Complete structured quality assurance, evaluation, and labeling tasks on GenAI outputs, following detailed written task instructions
  • Review AI-generated clinical content for accuracy, completeness, and adherence to defined criteria, documenting your reasoning clearly and consistently
  • Apply consistent judgment across a high volume of work without drifting from the rubric
  • Bring questions and blockers to the physician overseeing the work as they come up
  • Handle protected health information in accordance with HIPAA and our clinical safety and privacy standards

Required Qualifications:

  • 3+ months of experience as a medical scribe, or comparable medical educational or professional experience (for example: nursing, PA, or pre-med student; EMT, CNA, or medical assistant)
  • Working fluency in medical terminology and clinical documentation, and an understanding of how care is recorded in the chart
  • Exceptional attention to detail, and the ability to explain your reasoning clearly in writing
  • Reliable, consistent availability of 20–40 hours per week (full time preferred) across a 6 month commitment
  • Able to work onsite in Palo Alto 3 days/week

Desirable Skills and Experience:

  • Familiarity with EHR systems (Epic, Cerner) and clinical documentation workflows
  • Experience with chart abstraction, data labeling or annotation, quality improvement, or clinical research
  • Interest in AI/ML in healthcare and in how clinical AI is evaluated for safety and accuracy

Why Join Qualified Health?

You will be mentored by our physicians, who bring their clinical training to building products that change how care is delivered. You will sit with the clinical team, your findings will influence what ships, and there is opportunity to extend beyond the initial 6 months. Our physicians are glad to write letters of recommendation for interns they've worked closely with.

This is an opportunity to join a fast-growing company and a world-class team that is poised to change the healthcare industry. We are a passionate, mission-driven team that is building a category-defining product, and we are backed by premier investors.

Our employees are integral to achieving our goals, so we are proud to offer competitive pay, flexible working hours, hybrid work options, and an inclusive environment that fosters creativity and innovation.

Our Commitment to Diversity

Qualified Health is an equal opportunity employer. We believe that a diverse and inclusive workplace is essential to our success, and we are committed to building a team that reflects the world we live in. We encourage applications from all qualified individuals, regardless of race, color, religion, gender, sexual orientation, gender identity or expression, age, national origin, marital status, disability, or veteran status.

Pay & Details:

This is a paid, hourly internship beginning in September 2026, at $40.00 per hour. The role runs for 6 months at 20–40 hours per week, with full time preferred, and may be extended based on performance and team needs.

Join our mission to revolutionize healthcare with AI. To apply, please send your resume through the application below.

Source: Qualified Health Pbc careers (Ashby)

Similar roles