Site Reliability Engineer, AI Infrastructure (Starshield)

SpaceX
Palo Alto, CA
On-site

Who this role is best for

Best suited to candidates with experience in GPU infrastructure and national security domains working in high-stakes, classified environments.

Best fit for

  • Candidates with experience deploying and managing Kubernetes/AI clusters in classified settings
    — “Develop automation to deploy and manage on-premise Kubernetes\AI clusters”
  • Engineers with a background in national security or government infrastructure projects
    — “Starshield is the world’s largest US government satellite constellation”
  • Professionals skilled in scripting and infrastructure automation for high availability systems
    — “Monitoring and alerting supporting systems to have high availability”

Things to consider

  • This role requires obtaining a Top Secret clearance, which may take significant time and effort
    — “This position requires successfully obtaining and maintaining a Top Secret Security Clearance”
  • Candidates must be U.S. citizens to meet ITAR export regulations
    — “applicant must be a (i) U.S. citizen”
  • The role may involve extended hours and global travel, which could affect work-life balance
    — “Must be willing to work extended hours and weekends as needed”

How to stand out

  • Highlight experience with large-scale AI cluster design and deployment in your resume
    — “Design, validate, and productize solutions for AI clusters (100k+ GPU scale)”
  • Emphasize your ability to manage on-premise infrastructure and automation tools
    — “Develop automation to deploy and manage on-premise compute resources”
  • Showcase your experience with GPU platforms like NVIDIA Blackwell/Rubin
    — “Experience with NVIDIA GPU deployment stacks (Blackwell/Rubin)”
  • Demonstrate your ability to collaborate with AI engineers on scalable products
    — “Closely collaborate with AI engineers to create highly scalable, operable, and maintainable products”
  • Include examples of managing distributed databases and monitoring systems
    — “Deploy and manage core infrastructure such as databases, monitoring and distributed storage”
Pace · Fast PacedCollaboration · HighAutonomy · MediumDecision Impact · Team

Derived from job-description analysis by Serendipath's career intelligence engine.

What success looks like

  • Deployed and managed on-premise Kubernetes/AI clusters
  • Developed automation to deploy and manage on-premise compute resources
  • Created highly scalable and maintainable software products
Typical background
Bachelor’s degree in computer science, information systems/IT, or an engineering discipline1+ years of professional experience in site reliability engineering or DevOps

Skills & requirements

Required

Site_reliability_engineeringLinux_operating_systemsTerraformAnsibleContainerization_technologiesBash_scriptingPython_developmentKubernetesDatabase_managementMonitoring_and_alertingHigh_availabilityPerformance_bottlenecksDistributed_databasesData_modeling

Preferred

Continuous_integrationBuild_technologiesPerformance_improvement_techniques

Stack & domain

Site Reliability EngineeringDevOpsLinux Operating SystemsTerraformAnsibleContainerization TechnologiesBashPythonC++GoKubernetesDistributed DatabasesData ModelingCollaborationProblem SolvingInnovationSystem AvailabilityMonitoringAlertingAI InfrastructureNational Security

About the role

Original posting from SpaceX via Greenhouse

SpaceX was founded under the belief that a future where humanity is out exploring the stars is fundamentally more exciting than one where we are not. Today SpaceX is actively developing the technologies to make this possible, with the ultimate goal of enabling human life on Mars.

SITE RELIABILITY ENGINEER (STARSHIELD)

At SpaceX we’re leveraging our experience in building rockets and spacecraft to deploy the Starshield constellation. Starshield is the world’s largest US government satellite constellation and is tasked with providing immediate access to critical intelligence and national security data for the US government anywhere on the globe. We design, build, test, and operate all parts of the system – receivers that allow users to connect within minutes, and the software that brings it all together. We’ve only begun to scratch the surface of Starshield's global impact and are looking for best-in-class engineers to help us further our ambitious goals.

As an engineer focused on Starshield's software and GPU infrastructure, you will design, operate and scale the infrastructure that supports critical national security missions. These positions cover a variety of areas ranging from Site Reliability Engineering, Developer Operations, and GPU platforms. You will develop automation to deploy and manage on-premise compute resources, create highly scalable and maintainable software products, and directly collaborate with engineering across the board.

RESPONSIBILITIES: 

Manage GPU/CPU infrastructure deployments to Top Secret data centers

Manage and provide support for GPU as a service for external customers on bare metal hardware and virtualized platforms

Design, validate, and productize solutions for AI clusters (100k+ GPU scale)

Develop automation to deploy and manage on-premise Kubernetes\AI clusters, and operating systems

Deploy and manage core infrastructure such as databases, monitoring and distributed storage

Closely collaborate with AI engineers to create highly scalable, operable, and maintainable products

Engage in and improve the whole lifecycle of services -- from inception and design, through deployment, operation and refinement

Monitoring and alerting supporting systems to have high availability

Identify areas for improvement and create innovative solutions that enable high system availability

BASIC QUALIFICATIONS:

Bachelor’s degree in computer science, information systems/IT, or an engineering discipline and 1+ years of professional experience in site reliability engineering or DevOps; OR 3+ years of professional experience in site reliability engineering or DevOps in lieu of a degree

1+ years of professional experience with Linux operating systems

Experience with Terraform, Ansible, or other infrastructure tools

Experience with containerization technologies (i.e. OCI containers, Kubernetes)

Experience scripting in Bash, Python, or other similar languages

Development experience in Python, C++, or Go

PREFERRED SKILLS AND EXPERIENCE:

1+ years of experience with Python and Python-based development frameworks

Experience managing Kubernetes clusters, not just using them

Knowledge of Linux boot process and systems configuration

Deep understanding of testing, continuous integration, build, deployment & continuous monitoring

Understanding of relevant build technologies, such as Bazel and Makefiles

Focus on performance bottlenecks and performance improvement techniques

Understanding of distributed databases and data modeling

Experience with automatically managing thousands of servers (eg: Terraform or Ansible)

Strong networking knowledge of TCP/IP

Cloud virtualization experience (as the cloud provider)

Experience using NVIDIA GPU deployment stacks (Blackwell/Rubin)

Excellent communications skills with the ability to communicate with customers, peers, management etc. in both formal and informal situations

Active Top Secret, Top Secret SCI, or DOE Level Q clearance

ADDITIONAL REQUIREMENTS:

Must be willing to work extended hours and weekends as needed

Must be willing to travel domestically and globally in the future when needed

This position requires successfully obtaining and maintaining a Top Secret Security Clearance as a condition of employment. While the clearance may not be immediately necessary upon hire, we encourage you to initiate the application process promptly upon accepting this offer. Your ability to secure the necessary clearance is essential for fulfilling key responsibilities of the role. Should you be unable to obtain it, SpaceX reserves the right to modify or terminate your employment to align with operational needs.

COMPENSATION AND BENEFITS:

Pay Range:

Level 1: $125,000.00 - $160,000.00

Level 2: $145,000.00 - $195,000.00

Your actual level and base salary will be determined on a case-by-case basis and may vary based on the following considerations: job-related knowledge and skills, education, and experience.

Base salary is just one part of your total rewards package at SpaceX. You may also be eligible for long-term incentives, in the form of company stock or long-term cash awards, as well as potential discretionary bonuses and the ability to purchase additional stock at a discount through an Employee Stock Purchase Plan. You will also receive access to comprehensive medical, vision, and dental coverage, access to a 401(k) retirement plan, short and long-term disability insurance, life insurance, paid parental leave, and various other discounts and perks. You may also accrue 3 weeks of paid vacation and will be eligible for 10 or more paid holidays per year. Employees accrue paid sick leave pursuant to Company policy which satisfies or exceeds the accrual, carryover, and use requirements of the law.

Those with an active clearance will receive a 10% differential, up to an additional $20,000 annually, once officially briefed into a classified program.

ITAR REQUIREMENTS:

To conform to U.S. Government export regulations, applicant must be a (i) U.S. citizen or national, (ii) U.S. lawful, permanent resident (aka green card holder), (iii) Refugee under 8 U.S.C. § 1157, or (iv) Asylee under 8 U.S.C. § 1158, or be eligible to obtain the required authorizations from the U.S. Department of State. Learn more about the ITAR here.  

SpaceX is an Equal Opportunity Employer; employment with SpaceX is governed on the basis of merit, competence and qualifications and will not be influenced in any manner by race, color, religion, gender, national origin/ethnicity, veteran status, disability status, age, sexual orientation, gender identity, marital status, mental or physical disability or any other legally protected status.

Applicants wishing to view a copy of SpaceX’s Affirmative Action Plan for veterans and individuals with disabilities, or applicants requiring reasonable accommodation to the application/interview process should reach out to EEOCompliance@spacex.com. 

Source: SpaceX careers (Greenhouse)

Similar roles