Applied Scientist / Research Engineer — Data Quality & Evaluation

Maxinsights
Santa Clara, US
On-site

Who this role is best for

Data scientists with ML research experience and a focus on data-centric AI will find this data quality and evaluation role ideal.

Best fit for

  • Candidates with experience in data curation and model evaluation methods
    — “training data curation, dataset optimization, model evaluation, benchmark development”
  • Individuals skilled in Python and ML frameworks who prefer building tools over labeling data
    — “Strong Python skills and experience with relevant ML frameworks and data processing tools”
  • Those with a background in applied research and a history of translating findings into technical solutions
    — “Experience developing technical solutions, research methods, or automated pipelines rather than only performing annotation or data labeling”

Things to consider

  • The role demands a focus on data-centric AI, not just model training
    — “improve the quality of training data and evaluation systems used to develop advanced AI models”

How to stand out

  • Highlight experience in dataset optimization and evaluation framework design in your resume
    — “Design evaluation frameworks and benchmarks to measure model capabilities and identify performance gaps”
  • Emphasize projects where you built automated data pipelines or used active learning techniques
    — “Build automated tools and pipelines to improve data quality while reducing annotation costs and manual effort”
  • Demonstrate your ability to analyze model failures and link them to data characteristics
    — “Analyze model failures to understand how training data affects model behavior and generalization”
Pace · SteadyCollaboration · HighAutonomy · MediumDecision Impact · Team

Derived from job-description analysis by Serendipath's career intelligence engine.

What success looks like

  • improving data quality and model performance
  • developing automated tools for data quality
Typical background
strong background in applied ML research or data-centric AI

Skills & requirements

Required

Applied ML ResearchData CurationModel EvaluationActive LearningHuman-in-the-loop Evaluation

Preferred

AI Data Or Model Development ExperienceBuilding Evaluation Frameworks

Stack & domain

Applied ML ResearchData-centric AIModel EvaluationPythonML FrameworksData Processing ToolsActive LearningHuman-in-the-loop EvaluationData CurationData ScoringData MixingEvaluation FrameworksBenchmark DevelopmentModel Failures AnalysisAutomated ToolsPipelinesResearchAnalyticalCommunicationCollaborationAIData QualityEvaluation Systems

About the role

Original posting from Maxinsights via Ashby

JOB DESCRIPTION:

JOB OVERVIEW

We are looking for an Applied Scientist/Research Engineer to improve the quality of training data and evaluation systems used to develop advanced AI models.

In this role, you will investigate how data quality, dataset composition, and evaluation methods influence model performance. You will design experiments, develop data curation and scoring methods, analyze model failures, and build tools that help us identify the data most valuable for training and improving AI systems.

This role is ideal for someone with a strong background in applied ML research, data-centric AI, or model evaluation who enjoys turning research insights into practical improvements.

WHAT YOU WILL OWN

  • Develop methods for training data curation, quality scoring, dataset selection, and data mixing.
  • Design evaluation frameworks and benchmarks to measure model capabilities and identify performance gaps.
  • Analyze model failures to understand how training data affects model behavior and generalization.
  • Apply techniques such as active learning and human-in-the-loop evaluation to improve data efficiency and annotation quality.
  • Build automated tools and pipelines to improve data quality while reducing annotation costs and manual effort.
  • Design and run experiments to measure how data improvements translate into better model performance.
  • Collaborate with research, engineering, and data operations teams to turn findings into scalable solutions.

REQUIRED QUALIFICATIONS

  • Experience in applied ML research, data science, AI evaluation, or a related technical field.
  • Strong Python skills and experience with relevant ML frameworks and data processing tools.
  • Experience with one or more of the following: training data curation, dataset optimization, model evaluation, benchmark development, active learning, or post-training.
  • Strong experimental design and analytical skills, with the ability to connect data characteristics to model performance.
  • Experience developing technical solutions, research methods, or automated pipelines rather than only performing annotation or data labeling.
  • Ability to communicate research findings clearly and translate them into actionable improvements.

PREFERRED QUALIFICATIONS

  • Experience at AI data or model development companies (ex. Scale AI, Mercor, Surge AI, or similar organizations).
  • Experience improving model performance through better training data or dataset composition.
  • Experience building evaluation frameworks or analyzing model failures.
  • Experience with human-in-the-loop systems, annotation quality measurement, or data selection algorithms.
  • Research publications or demonstrated applied research contributions are a plus.

DEFAULT BENEFITS:

  • Health insurance
  • Vision care
  • Dental coverage
  • 401(k)
  • Paid holidays
  • PTO (Paid Time Off)
  • Sick leave

Source: Maxinsights careers (Ashby)

Similar roles