Geared toward experienced ML practitioners comfortable with large-scale distributed training and LLM fine-tuning, this role emphasizes collaboration with science and engineering teams in a remote Canadian setting.
Derived from job-description analysis by Serendipath's career intelligence engine.
Original posting from Robots and Pencils via Greenhouse
Staff ML Data Scientist, LLM Recommender Systems
We’re looking for a top-tier ML Data Scientist to build, train, and deploy next-generation LLM-based recommender systems. You will work directly with a top notch science team, the AWS account team, and the AWS Applied Scientist leading the science. The role covers the full path from raw customer data to a deployed model. We are looking for a seasoned applied scientist with a multi-year record of production ML, not someone who recently moved over from analytics or data science generalism.
Why This Role Matters
At Robots & Pencils, we design AI systems for a human world. Our name says it all. Robots and pencils means engineering paired with creativity, because every system we ship has to work for real people in real workflows. Here, you shape how AI integrates into enterprise operations and how products create measurable impact for clients and the people they serve. We ship production-ready AI in 30 to 45 days. That pace demands people who take ownership, lead with craft, and care deeply about what they put their name on.
What You’ll Do
Turn structured and unstructured traveler and catalog data into narrative text documents and synthetic data, at a scale of tens of millions of documents
Make the feature and context engineering decisions: what information goes into each document and how to express it (e.g., market-relative price, tier, trip context)
Run large-scale distributed training on Amazon SageMaker AI, including HyperPod
Fine-tune models with LoRA/QLoRA adapters, supervised training, and reinforcement learning for ranking and reasoning
Own deep learning modeling, including architecture, ranking heads, and co-training and multi-task setups
Deploy low-latency, cost-efficient inference, including quantization, distillation, and profiling
Build evaluation sets, run ablations, catch data leakage, and write up findings clearly for Expedia and the AWS science team
Handle customer data safely, including personal-data removal, working inside the customer’s environment
Bring an AI-forward mindset to your daily work, using tools like Claude and Cursor to ship higher-quality work at pace
What You’ll Bring
Spark (PySpark) and the Hadoop ecosystem (Hive, HDFS) on corpora of tens of millions of documents
Hands-on Amazon SageMaker AI: processing, training, distributed training (HyperPod), and deployment
LLM fine-tuning (LoRA/QLoRA) and reinforcement learning (e.g., GRPO or PPO-style methods)
Strong deep learning fundamentals in PyTorch: architectures, training dynamics, co-training
Large-scale batch and production LLM inference (e.g., vLLM, Amazon Bedrock batch)
Data privacy and security on AWS: personal-data removal, IAM, KMS, Lake Formation
Clear written communication with customer and science stakeholders
Demonstrable usage of AI-forward tools such as Claude and Cursor
Helpful Extras and Unique Skills
AWS Trainium or Inferentia (Neuron SDK)
Recommender systems: ranking metrics (NDCG, MRR), sequential and generative recommenders
Embedding quantization or semantic IDs
Customer-facing or embedded consulting experience