Expert Talent Pool, Feedback and Alignment (Senior Expert)

Clera
Worldwide Remote
RemoteCareer-pivot friendly

Who this role is best for

Geared toward professionals with deep domain expertise and experience in rating or reviewing expert-level work, comfortable with fact-checking and calibrating standards across specialized fields.

Best fit for

  • Candidates with 5+ years of experience in grading or reviewing expert answers and a background in a specialized field like law or medicine
    — “At least 5 years of professional experience in rating, review, editing, fact-checking, trust and safety, or a relevant professional domain.”
  • Individuals with leadership experience in grading teams and a track record of refining evaluation standards
    — “Experience leading or reviewing other raters, such as as a QA lead, rubric owner, or reviewer of reviewers.”
  • Professionals with advanced degrees or licenses in their domain and strong written English skills
    — “An advanced degree, professional license, or deep working expertise in the domain you grade. Native or near-native written English proficiency.”

Things to consider

  • The role demands a high level of attention to detail and the ability to work across multiple specialized domains.
    — “Evaluate steps in model reasoning across fields such as mathematics, coding, law, medicine, and finance.”
  • Candidates must be prepared to engage in live sessions and provide real-time feedback.
    — “Assess search relevance and provide feedback in live sessions.”

How to stand out

  • Highlight your background in refining grading rubrics and your experience leading review teams.
    — “Review other raters' work, calibrate standards, and develop or refine grading guidelines.”
  • Showcase your written English proficiency, especially in creating clear and detailed rationales.
    — “Native or near-native written English proficiency.”
Pace · SteadyCollaboration · MediumAutonomy · MediumDecision Impact · Individual

Derived from job-description analysis by Serendipath's career intelligence engine.

What success looks like

  • Evaluate complex AI model outputs and help calibrate other graders
Typical background
At least 5 years of professional experience in rating, review, editing, fact-checking, trust and safety, or a relevant professional domain

Skills & requirements

Required

RatingReviewingEditingFact-checkingTrust And SafetyAI Data AnnotationSearch-quality RatingWriting And Maintaining Grader Guidelines

Preferred

Leading Or Reviewing Other RatersQA LeadRubric OwnerReviewer Of Reviewers

Stack & domain

RatingReviewingFact-checkingTrust And SafetyAI Data AnnotationSearch-quality RatingGrader Guidelines Writing And MaintainingNative Or Near-native Written English ProficiencyExperience Grading Or Reviewing Expert-level AnswersLeading Or Reviewing Other RatersQA LeadRubric OwnerReviewer Of ReviewersAI Model Outputs EvaluationComplex Claims Fact-checkingContent Policies AssessmentSearch Relevance AssessmentAI Development

About the role

Original posting from Clera via Ashby

ABOUT THE ROLE

Join a remote expert network as a senior rater and reviewer, evaluating complex AI model outputs and helping calibrate other graders. Your work supports the development of high-quality, trustworthy AI systems across specialized fields.

WHAT YOU'LL DO

  • Rank and rate expert-level model answers using detailed rubrics and write clear rationales.
  • Evaluate steps in model reasoning across fields such as mathematics, coding, law, medicine, and finance.
  • Fact-check complex claims against primary sources and verify accuracy.
  • Assess answers against content policies, style guides, and edge cases.
  • Review other raters' work, calibrate standards, and develop or refine grading guidelines.
  • Assess search relevance and provide feedback in live sessions.

WHAT WE'RE LOOKING FOR

  • At least 5 years of professional experience in rating, review, editing, fact-checking, trust and safety, or a relevant professional domain.
  • Native or near-native written English proficiency.
  • Experience grading or reviewing expert-level answers against rubrics, policies, or style guides.
  • Experience leading or reviewing other raters, such as as a QA lead, rubric owner, or reviewer of reviewers.
  • An advanced degree, professional license, or deep working expertise in the domain you grade.
  • Hands-on AI data annotation experience, search-quality rating, or experience writing and maintaining grader guidelines is a plus.

COMPENSATION & BENEFITS

Compensation is $50 to $120 per hour, depending on the project, field, and expertise level. Payment is based on hours worked.

LOCATION

Fully remote. Projects typically run from one week to several months.

Source: Clera careers (Ashby)

Similar roles

More roles at Clera