Member of Technical Staff, Post-Training

Handshake

San Francisco, CA

Hybrid

Who this role is best for

Candidates with post-training model optimization experience and systems-building expertise will find this high-ownership role in AI infrastructure development at Handshake.

Best fit for

Candidates with post-training model optimization experience and systems-building expertise
— “We care more about demonstrated research capability, technical judgment, and a builder’s mindset than a specific title, degree, or career path.”
Individuals who can translate ambiguous needs into technical systems and experiments
— “Translate open-ended research or partner needs into clear hypotheses, experiments, evaluation plans, and production-quality implementations.”
Researchers or engineers with a track record of productizing AI systems into reusable platforms
— “Experience productizing research or repeated customer work into robust, reusable platforms.”

Things to consider

Location preference is strong for San Francisco and Mountain View despite remote flexibility
— “Location: San Francisco & Mountain View preferred; we are open to exceptional candidates in other locations.”
The role demands a high level of autonomy and ownership in a rapidly evolving space
— “Early members of the team will have unusual influence over our technical direction, operating culture, and the reusable systems we build.”

How to stand out

Highlight experience with model evaluation, preference optimization, and data pipelines in your resume
— “Design post-training systems and methodologies for frontier models, including supervised fine-tuning, reinforcement learning, preference optimization, reward modeling, and related approaches.”
Emphasize your ability to work with researchers and domain experts on ambiguous problems
— “Partner directly with AI researchers and domain experts to develop high-signal data, feedback, and evaluation methods.”
Showcase your ability to build scalable systems, not just execute on specifications
— “Experience designing systems—not only implementing specifications—including the ability to make tradeoffs around quality, scale, reliability, and reuse.”
Include evidence of technical leadership in open-source or published research
— “Published research, meaningful open-source contributions, or evidence of technical leadership in ML systems or AI research.”

Pace · Fast PacedCollaboration · HighAutonomy · HighDecision Impact · Company

Derived from job-description analysis by Serendipath's career intelligence engine.

What success looks like

post-training systems
evaluation frameworks
data pipelines
model improvement
reusable systems

Typical background

research scienceresearch engineeringmachine learning engineering

Skills & requirements

Required

Post-training SystemsModel EvaluationData PipelinesMachine Learning EngineeringResearch EngineeringReinforcement LearningSupervised Fine-tuningReward Modeling

Preferred

Start-up ExperienceAerospace ExperienceAudit SupportPre-ipo Environments

Stack & domain

Research ScienceResearch EngineeringMachine Learning EngineeringSupervised Fine-tuningReinforcement LearningPreference OptimizationReward ModelingDemonstrated Research CapabilityTechnical JudgmentBuilder’s MindsetAIData BusinessResearch PlatformsCustomer-facing AI Systems

About the role

Original posting from Handshake via Ashby

ABOUT HANDSHAKE
Handshake was founded on a simple belief that everyone deserves a path to a great career, regardless of where they went to school or who they know. Today, we power 25 million job seekers, 1 million+ employers, and 1,600 educational institutions.
In 2025, we started Handshake AI and built the fastest-growing AI data business in history. We work directly with frontier AI lab researchers to create evaluations, publish benchmarks, and push the boundary of data. We've grown from $0 to ~$1B run rate and pay ~$60M to over 30K individuals every month.
ABOUT HANDSHAKE LABS
Handshake Labs is building external AI products, research platforms, and customer-facing AI systems. We are evolving work that is often custom-built for an individual partner into reusable products and platforms that improve with every deployment.
Our work spans the full post-training loop: designing evaluations and training environments, building high-quality data and feedback systems, running experiments, and turning what works into durable infrastructure. For example, we are developing agents that can analyze long, complex coding-agent sessions in days rather than weeks—with expert review and calibration built into the system.
THE ROLE
We are hiring a Member of Technical Staff, Post-Training to help define and build this new organization. This is a broad, high-ownership role for researchers who build. You may come from research science, research engineering, machine learning engineering, or a closely related background; what matters is the ability to reason deeply about model improvement and turn that reasoning into reliable systems.
You will partner with researchers, domain experts, and customers to turn ambiguous post-training questions into experiments, evaluation frameworks, data pipelines, and products. Early members of the team will have unusual influence over our technical direction, operating culture, and the reusable systems we build.
We care more about demonstrated research capability, technical judgment, and a builder’s mindset than a specific title, degree, or career path.
Location: San Francisco & Mountain View preferred; we are open to exceptional candidates in other locations.
WHAT YOU’LL DO
Design post-training systems and methodologies for frontier models, including supervised fine-tuning, reinforcement learning, preference optimization, reward modeling, and related approaches.
Translate open-ended research or partner needs into clear hypotheses, experiments, evaluation plans, and production-quality implementations.
Build and improve evaluation frameworks, benchmarks, training environments, data-processing pipelines, and quality-control systems.
Run fast, rigorous iteration loops: prototype, evaluate, interpret results, and turn learnings into the next system or product.
Partner directly with AI researchers and domain experts to develop high-signal data, feedback, and evaluation methods.
Identify repeatable patterns across engagements and productize them into reusable software and platforms.
Raise the technical bar through strong design judgment, clear communication, code quality, and mentorship.
Contribute to the field through benchmarks, open-source tools, research, and technical writing where it creates leverage.
WHAT WE’RE LOOKING FOR
3+ years of demonstrated strength in post-training, fine-tuning, or model-evaluation work. Relevant experience may include RL, SFT, LoRA/PEFT, full fine-tuning, RLHF, DPO, PPO, reward modeling, or training environments.
Strong Python skills and the ability to write clean, efficient, scalable software.
Hands-on experience with modern ML tooling, particularly PyTorch and large-scale data, training, or evaluation workflows.
Sound experimental judgment: you can form hypotheses, choose meaningful metrics, diagnose failures, and distinguish signal from noise.
Experience designing systems—not only implementing specifications—including the ability to make tradeoffs around quality, scale, reliability, and reuse.
Comfort operating in an ambiguous, fast-moving environment with substantial ownership.
Collaborative, low-ego communication and the ability to work effectively with researchers, engineers, domain experts, and customers.
ESPECIALLY COMPELLING EXPERIENCE
Building or operating large-scale ML training, inference, data, or evaluation systems.
Developing LLM/agent benchmarks, evaluation methodologies, annotation systems, or data-quality frameworks.
Research or applied work on reinforcement learning, alignment, model behavior, synthetic data, or human-in-the-loop systems.
Published research, meaningful open-source contributions, or evidence of technical leadership in ML systems or AI research.
Experience productizing research or repeated customer work into robust, reusable platforms.
WHY JOIN
Work on problems at the center of how frontier AI systems improve, alongside leading labs and domain experts.
Help build an early technical organization where your work shapes the roadmap, standards, and culture.
Move fluidly from research insight to real-world systems, with the resources and customer context to see those systems matter.
Join a company building durable infrastructure for careers in the AI economy.
PERKS
Handshake delivers benefits that help you feel supported—and thrive at work and in life.
The below benefits are for full-time US employees.
🎯 Ownership: Equity in a fast-growing company
💰 Financial Wellness: 401(k) match, competitive compensation, financial coaching
🍼 Family Support: Paid parental leave, fertility benefits, parental coaching
💝 Wellbeing: Medical, dental, and vision, mental health support, $500 wellness stipend
📚 Growth: $2,000 learning stipend, ongoing development
💻 Office: Commuting support, free lunch, and gym in our SF office
🏝 Time Off: Flexible PTO, 15 holidays + 2 flex days
🤝 Connection: Team outings & referral bonuses

Source: Handshake careers (Ashby)