Copy of Research Scientist / Engineer – Training Infrastructure

Luma

London, United Kingdom

Hybrid

Who this role is best for

Candidates with distributed systems experience and foundation model training expertise will find this role in London, UK, with a focus on PyTorch and GPU cluster optimization.

Best fit for

Candidates with experience designing distributed training systems at scale and optimizing foundation models for GPU clusters.
— “Design, implement, and optimize efficient distributed training systems for models across thousands of GPUs.”
Engineers who have tackled advanced parallelism techniques like FSDP and multi-node training in real-world scenarios.
— “If you haven't worked at the level of FSDP and multi-node training, this is the wrong depth.”
Individuals with a strong background in GPU cluster networking, storage, and communication libraries.
— “Deep understanding of GPU clusters, networking, and storage systems.”

Things to consider

This role requires a high level of technical depth in distributed systems and foundation model training.
— “This is hard PyTorch, CUDA, and distributed-systems work — advanced parallelism, training stability, and utilization across massive clusters.”
The role emphasizes a hands-on approach to solving real problems in large-scale training environments.
— “It fits an engineer who's solved real problems training foundation models at scale.”

How to stand out

Highlight specific contributions to training stability or resource optimization in large-scale systems.
— “Optimize training stability, convergence, and resource utilization across massive clusters.”
Demonstrate experience with FSDP, Tensor Parallel, and similar advanced parallelism techniques.
— “Research and implement advanced parallelization (FSDP, Tensor Parallel, Pipeline Parallel, Expert Parallel).”
Showcase your ability to build tools for monitoring and debugging large training runs.
— “Build monitoring, visualization, and debugging tools for large-scale training runs.”

Pace · Fast PacedCollaboration · HighAutonomy · HighDecision Impact · Team

Derived from job-description analysis by Serendipath's career intelligence engine.

What success looks like

efficient distributed training systems
advanced parallelization
training stability

Typical background

distributed PyTorch trainingfoundation-model training

Skills & requirements

Required

Distributed SystemsPyTorchCUDAFSDPTensor ParallelPipeline ParallelExpert ParallelMonitoringVisualizationDebugging

Preferred

Linux Systems AdministrationContainerizationOrchestrationCloud Infrastructure

Stack & domain

PyTorchCUDAFSDPTensor ParallelPipeline ParallelExpert ParallelNCCLMPILinux Systems AdministrationScriptingContainerizationOrchestrationCloud InfrastructureProblem-solvingInnovationTeamworkAIMachine LearningDistributed SystemsTraining Infrastructure

About the role

Original posting from Luma via Ashby

You'll build the distributed systems that train Luma's large-scale multimodal models across thousands of GPUs, so researchers can focus on innovation on top of reliable, efficient, scalable infrastructure.
This is hard PyTorch, CUDA, and distributed-systems work — advanced parallelism, training stability, and utilization across massive clusters. It fits an engineer who's solved real problems training foundation models at scale. If you haven't worked at the level of FSDP and multi-node training, this is the wrong depth.
What You'll Own
Design, implement, and optimize efficient distributed training systems for models across thousands of GPUs.
Research and implement advanced parallelization (FSDP, Tensor Parallel, Pipeline Parallel, Expert Parallel).
Build monitoring, visualization, and debugging tools for large-scale training runs.
Optimize training stability, convergence, and resource utilization across massive clusters.
First 90 Days
One way the first 90 could unfold.
Days 1–30 — Immerse & Diagnose: Learn the current training stack and where stability and utilization hurt at scale.
Days 30–60 — Ship & Validate: Land a parallelization or stability improvement that measurably helps a real training run.
Days 60–90 — Scale & Systemize: Build the monitoring and tooling that keeps large runs reliable and efficient.
What You Bring
Extensive distributed PyTorch training and parallelisms in foundation-model training.
Deep understanding of GPU clusters, networking, and storage systems.
Familiarity with communication libraries (NCCL, MPI) and distributed-system optimization.
Nice to Have
Strong Linux systems administration and scripting.
Experience managing training runs across 100+ GPUs.
Experience with containerization, orchestration, and cloud infrastructure.
About Luma: Luma's mission is to build unified general intelligence that can generate, understand, and operate in the physical world. We believe multimodality is critical for intelligence — the next step beyond language models comes from vision. Luma is an equal opportunity employer.

Source: Luma careers (Ashby)