Software Engineer, Inference

Luma

Redwood City +1 more

Hybrid

Who this role is best for

Systems engineers with model-serving experience and Kubernetes expertise will find this role in large-scale inference systems work.

Best fit for

Candidates with experience in deploying models using PyTorch or similar frameworks and optimizing GPU utilization at scale.
— “Experience deploying models with PyTorch, Hugging Face, vLLM, SGLang, TensorRT-LLM, or similar.”
Individuals who have worked on scheduling systems, fleet management, and high-performance ML infrastructure with 100+ GPUs.
— “High-performance large-scale ML systems (100+ GPUs).”
Professionals comfortable with Linux, Docker, Kubernetes, and building deployment pipelines for model checkpoints.
— “Experience with Linux, Docker, and Kubernetes, and with orchestration, deployment, and scheduling.”

Things to consider

This role requires managing and optimizing inference workloads across multiple hardware providers and clusters.
— “Manage and optimize inference workloads across clusters and hardware providers”
Candidates must be prepared to work on both internal tooling and external deployment pipelines.
— “Build internal tooling to measure, profile, and track the lifetime of inference jobs and workflows.”

How to stand out

Highlight experience with large-scale Kubernetes deployments and model-serving optimization in your resume.
— “It fits a strong systems engineer comfortable with model serving and Kubernetes at scale.”
Emphasize your ability to build and maintain scheduling systems that meet strict SLOs.
— “Build scheduling systems that use expensive GPU resources optimally while meeting SLOs”
Showcase your background in automating and testing inference services for high uptime.
— “Automate, test, and maintain inference services for maximum uptime and reliability.”

Pace · SteadyCollaboration · MediumAutonomy · MediumDecision Impact · Team

Derived from job-description analysis by Serendipath's career intelligence engine.

What success looks like

integrating new model architectures
optimizing model efficiency
building internal tooling
automating and maintaining inference services
managing and optimizing inference workloads

Typical background

model servingKubernetesLinux

Skills & requirements

Required

Model ServingKubernetesLinuxDockerPyTorchHugging FaceVllmSglangTensorrt-llmQueuesSchedulingTraffic ControlFleet ManagementCI/CDGPU Resource ManagementSlosModel CheckpointsSdksInference EngineDeployment Pipelines

Preferred

Modern Networking StacksCUDAFfmpeg

Stack & domain

PythonSystem-architecturePyTorchHugging FaceVllmSglangTensorrt-llmQueuesSchedulingTraffic ControlFleet ManagementLinuxDockerKubernetesOrchestrationDeploymentRedisS3-compatible StorageRDMARoceInfinibandNvlinkCUDAFfmpegMultimedia ProcessingLeadershipCommunicationProblem-solvingTeamworkCustomer ObsessionTechnical Leadership

About the role

Original posting from Luma via Ashby

You'll own how Luma's models get served — integrating new architectures into the inference engine, scaling deployments across thousands of machines, and keeping expensive GPU fleets busy while meeting internal SLOs.
This is large-scale inference systems work: scheduling, fleet management, deployment pipelines, and reliability across clusters and hardware providers. It fits a strong systems engineer comfortable with model serving and Kubernetes at scale. If you want pure modeling rather than the systems that run models, this is firmly the systems side.
What You'll Own
Ship new model architectures by integrating them into the inference engine.
Collaborate across research, engineering, and infrastructure to optimize model efficiency and deployments.
Build internal tooling to measure, profile, and track the lifetime of inference jobs and workflows.
Automate, test, and maintain inference services for maximum uptime and reliability.
Manage and optimize inference workloads across clusters and hardware providers, and scale deployments across thousands of machines.
Build scheduling systems that use expensive GPU resources optimally while meeting SLOs, and maintain CI/CD for model checkpoints and SDKs.
First 90 Days
One way the first 90 could unfold.
Days 1–30 — Immerse & Diagnose: Learn the inference stack, the fleets, and where reliability or utilization break.
Days 30–60 — Ship & Validate: Integrate a model or ship tooling/scheduling that improves uptime or GPU utilization.
Days 60–90 — Scale & Systemize: Harden deployment pipelines and scheduling across clusters and providers.
What You Bring
Strong Python and system-architecture skills.
Experience deploying models with PyTorch, Hugging Face, vLLM, SGLang, TensorRT-LLM, or similar.
Experience with queues, scheduling, traffic control, and fleet management at scale.
Experience with Linux, Docker, and Kubernetes, and with orchestration, deployment, and scheduling.
Familiarity with Redis and S3-compatible storage.
Nice to Have
Modern networking stacks including RDMA (RoCE, InfiniBand, NVLink).
High-performance large-scale ML systems (100+ GPUs).
CUDA, and FFmpeg or multimedia processing.
About Luma: Luma's mission is to build unified general intelligence that can generate, understand, and operate in the physical world. We believe multimodality is critical for intelligence — the next step beyond language models comes from vision. Luma is an equal opportunity employer.

Source: Luma careers (Ashby)