Senior Staff Software Engineer, DC Infrastructure

Crusoe
San Francisco +2 more
On-site

Who this role is best for

Strong fit for senior software engineers with distributed systems expertise and AI infrastructure experience who thrive in collaborative environments and can work in San Francisco or Sunnyvale, CA.

Best fit for

  • Candidates with hands-on GPU diagnostics experience and automation tooling development skills
    — “Developing and implementing deep-level diagnostics and troubleshooting of hardware faults within GPU racks
  • Individuals who have led technical direction in large-scale distributed systems projects
    — “Ability to set the technical direction for a specific project and execute
  • Professionals with cloud platform experience (Kubernetes, IaC, GCP) and hardware vendor collaboration
    — “Expertise in distributed systems, reliability, and cloud platforms (Kubernetes, IaC, GCP etc.)

Things to consider

  • High responsibility for system availability and performance due to critical infrastructure ownership
    — “Own the deployment, monitoring, and operational support of developed tooling

How to stand out

  • Highlight GPU-specific automation projects and experience with NVIDIA/AMD hardware in your resume
    — “Developing troubleshooting and automation tooling for GPU platforms including NVIDIA A100, H200, GB200, B200 and AMD 350X / 355X
  • Emphasize leadership in deploying observability tooling for high-density compute systems
    — “Developing and implementing advanced diagnostic, observability, automation and repair tooling
  • Demonstrate proficiency in at least one language (Go/Python/Rust) with concrete examples
    — “Strength in at least one programming language - Go, Python, Java, Rust
Pace · Fast PacedCollaboration · HighAutonomy · MediumDecision Impact · TeamLevel · Senior

Derived from job-description analysis by Serendipath's career intelligence engine.

What success looks like

  • develop and implement diagnostic and troubleshooting tools
  • deploy and monitor GPU fleet
Typical background
experience in software engineering with a focus on AI infrastructure

Skills & requirements

Required

GPU Cluster ManagementNvidia GPU TroubleshootingAutomation ToolingAI Agent DevelopmentObservability Tooling

Preferred

Project Planning Skills

Stack & domain

GPU ClustersAutomationObservability ToolingNvidia A100H200Gb200B200AMD 350X355XPyTorchNvidia NCCLDirect Liquid Cooling Hardware SystemsProblem-solvingOpportunity-findingSense Of UrgencyTeamworkIndependenceAI InfrastructureGPU Fleet ManagementHigh Performance Computing

About the role

Original posting from Crusoe via Ashby

Crusoe is on a mission to accelerate the abundance of energy and intelligence. As the only vertically integrated AI infrastructure company built from the ground up, we own and operate each layer of the stack — from electrons to tokens — to power the world's most ambitious AI workloads. When you join Crusoe, you join a team that is building the future, faster.

We're in the midst of the greatest industrial revolution of our time. The demand for AI compute is boundless, and power is a bottleneck. We're solving that — with an energy-first approach that makes AI infrastructure better for the world and faster for the people innovating with AI.

We're looking for problem-solving, opportunity-finding teammates with a sense of urgency, who believe in the scale of our ambition and thrive on a path not fully paved — people who want to grow their careers alongside a team of experts across energy, manufacturing, data center construction, and cloud services.

If you want to do the most meaningful work of your career, help our customers and partners advance their AI strategies, and be part of a high-performing team that believes in each other, come build with us at Crusoe.

Crusoe’s Data Center Infrastructure Engineering (DCIE) team is fundamental to our mission of providing AI hardware and infrastructure as a service. The team provides infrastructure for Crusoe’s fleet GPU’s and data center. The team sits at the nexus of high performance computing and AI infrastructure as a service.

The DCIE team owns, deployment maintenance, observability, critical environment, and automation. The team builds, and maintains GPU clusters, develops automation for logical and physical maintenance, provision systems, and observability tooling.

ABOUT THE ROLE:

We are seeking a highly skilled and motivated Software Engineer to join Crusoe’s Data Center Infrastructure Engineering team. This position is focused on the development of software for the management of a fleet of GPU servers as well as the data centers that house those systems. The role focuses on the developing and implementing advanced diagnostic, observability, automation and repair tooling for high-performance GPU compute clusters.

The ideal new team member will be a hands-on problem solver who is comfortable working independently. The new team member will play a critical role in maintaining the health and scalability of Crusoe’s rapidly growing GPU fleet.

WHAT YOU’LL BE DOING:

  • Developing and implementing deep-level diagnostics and troubleshooting of hardware faults within GPU racks and high-density compute systems.
  • Developing troubleshooting and automation tooling for GPU platforms including NVIDIA A100, H200, GB200, B200 and AMD 350X / 355X.
  • Developing automation and AI agents for executing component-level diagnosis and remediation for failed or degraded hardware.
  • In conjunction with data center operations develop innovative tooling and AI agents for managing the critical environment.
  • Developing tooling for post-repair validation and testing tools such as burn-in, Pytorch, and NVIDIA NCCL to ensure system stability and performance.
  • Own the deployment, monitoring, and operational support of developed tooling, ensuring solutions maximize GPU fleet availability and performance to drive customer success.
  • Developing automation and operational tooling for facilities management power as well as direct liquid cooling hardware systems

WHAT YOU’LL BRING TO THE TEAM:

  • Software engineering experience.
  • The ability to identify a problem, rapidly develop a scalable solution and ship it.
  • Ability to lean in and assist team members working on critical or complex technical initiatives.
  • Ability to set the technical direction for a specific project and execute.
  • Expertise in distributed systems, reliability, and cloud platforms (Kubernetes, IaC, GCP etc.)
  • Strength in at least one programming language - Go, Python, Java, Rust.
  • Strong analytical and problem-solving skills.
  • Excellent communication and collaboration skills.
  • Ability to work independently and within a team

NICE TO HAVE:

  • Experience with Temporal and Kubernetes.
  • Experience working directly with hardware vendors.
  • Background in large-scale GPU fleet operations or hyperscale data center environments.

BENEFITS:

  • Industry competitive pay
  • Restricted Stock Units in a fast growing, well-funded technology company
  • Health insurance package options that include HDHP and PPO, vision, and dental for you and your dependents
  • Employer contributions to HSA accounts
  • Paid Parental Leave
  • Paid life insurance, short-term and long-term disability
  • Teladoc
  • 401(k) with a 100% match up to 4% of salary
  • Generous paid time off and holiday schedule
  • Cell phone reimbursement
  • Tuition reimbursement
  • Subscription to the Calm app
  • MetLife Legal
  • Company paid commuter benefit; $300 per month

Compensation Range

Compensation will be paid in the range of up to $250,000 - $300,000 + Bonus. Restricted Stock Units are included in all offers. Compensation to be determined by the applicants knowledge, education, and abilities, as well as internal equity and alignment with market data.

Crusoe is an Equal Opportunity Employer. Employment decisions are made without regard to race, color, religion, disability, genetic information, pregnancy, citizenship, marital status, sex/gender, sexual preference/ orientation, gender identity, age, veteran status, national origin, or any other status protected by law or regulation.

Source: Crusoe careers (Ashby)

Similar roles