Runtime Engineer

Normalcomputing
Mountain View, NY
Hybrid

Who this role is best for

Best suited to systems engineers with deep experience in hardware abstraction and runtime design working in AI and semiconductor infrastructure.

Best fit for

  • Candidates with experience in runtime development for non-standard hardware and a deep understanding of LLM inference workloads
    — “Understanding of LLM inference workloads and what they demand of a runtime”
  • Individuals who have owned complex systems components from design to production and driven cross-team decisions
    — “Track record of owning a complex systems component end to end, from design through production, and driving technical decisions across teams”
  • Engineers comfortable with building software for pre-silicon hardware and simulators
    — “Comfort building software for hardware that is still evolving, from simulation through bring-up”

Things to consider

  • The role requires working across multiple global locations, including Mountain View and New York
    — “We work as one team across New York City, Silicon Valley (Mountain View), London, Copenhagen, and Seoul”
  • Candidates must be prepared to work with a pre-silicon chip and evolving hardware platforms
    — “Our chip is pre-silicon, so you will build the runtime against our simulators and FPGA prototypes first”

How to stand out

  • Highlight your end-to-end ownership of complex systems components in your resume and interview responses
    — “Track record of owning a complex systems component end to end, from design through production”
  • Emphasize your hands-on experience with runtime or driver development for accelerators or non-standard hardware
    — “Experience building runtimes, drivers, or hardware abstraction layers for accelerators or other non-standard hardware”
  • Demonstrate your ability to manage low-level performance and concurrency in C++ or Rust
    — “Strong systems programming in C++ or Rust: memory management, concurrency, and low-level performance”
  • Showcase familiarity with open runtime designs like IREE's HAL or CUDA/ROCm internals
    — “Experience with open runtime designs such as IREE's HAL, or with CUDA or ROCm runtime internals”
  • Demonstrate experience integrating runtimes with inference servers such as vLLM
    — “Experience integrating a runtime with an inference server such as vLLM”
Pace · Fast PacedCollaboration · HighAutonomy · MediumDecision Impact · Team

Derived from job-description analysis by Serendipath's career intelligence engine.

What success looks like

  • Design and implement runtime
  • Manage hardware abstraction
  • Optimize performance
Typical background
runtime developmentdriver developmenthardware abstraction

Skills & requirements

Required

Runtime ArchitectureSystems ProgrammingMemory ManagementConcurrencyLLM Inference Workloads

Preferred

Open Runtime DesignsIn-memory ComputeIntegration With Inference Servers

Stack & domain

Runtime ArchitectureDevice ManagementMemory AllocationHost-device TransferExecutable LoadingCommand SubmissionHardware AbstractionDevice State ManagementPerformance OptimizationCross-team InterfacesTechnical LeadershipSystems ProgrammingC++RustMemory ManagementConcurrencyLow-level PerformanceLLM Inference WorkloadsOpen Runtime DesignsIree's HALCUDARocm Runtime InternalsIn-memory ComputeNon-von-neumann HardwareIntegration With Inference ServerVllmOwnershipTechnical Decision-makingTeam CollaborationCommunicationProblem-solvingAdaptabilityAttention To DetailInnovationAISemiconductor IndustryCritical AI InfrastructureSystems That Power Our World

About the role

Original posting from Normalcomputing via Ashby

NORMAL COMPUTING | BUILD WITH US

Normal is an applied AI company solving the hardest problems in AI and silicon. We build foundational hardware and software for the semiconductor industry, critical AI infrastructure, and the broader systems that power our world, in partnership with the world's most advanced institutions. We work as one team across New York City, Silicon Valley (Mountain View), London, Copenhagen, and Seoul.

THE ROLE

Normal's hardware runs the most memory-bound parts of large-model inference inside memory itself, rather than shuttling weights to a processor. The software that makes this programmable doesn't exist yet in any standard form. The runtime is the layer the rest of that software runs through: it is how compiled programs get loaded, scheduled, and executed on our hardware. You will own its design and much of its implementation, and set its technical direction.

This is a senior, hands-on role. Our chip is pre-silicon, so you will build the runtime against our simulators and FPGA prototypes first, behind one interface, so that software above the runtime doesn't change when silicon arrives.

WHAT YOU WILL OWN

  • Runtime Architecture: Own the design of the runtime: device management, memory allocation and host-device transfer, executable loading, and command submission. Success means the compiler and serving layers program our hardware through one stable interface.
  • Hardware Abstraction: Hide the differences between simulators, FPGA prototypes, silicon, and later hardware generations behind that interface.
  • Device State: Manage the state that lives on our hardware between requests, such as resident model weights, so the serving layer can rely on it.
  • Performance: Keep runtime overhead off the critical path for LLM inference, measured by end-to-end latency and throughput.
  • Cross-Team Interfaces: Define the boundary with the compiler above the runtime, and with the driver and firmware below it, which our hardware team owns.
  • Technical Leadership: Set the runtime's technical direction, review designs that depend on it, and work with compiler and serving engineers so the interface serves them.

WHAT MAKES YOU A GREAT FIT

  • Experience building runtimes, drivers, or hardware abstraction layers for accelerators or other non-standard hardware
  • Strong systems programming in C++ or Rust: memory management, concurrency, and low-level performance
  • Track record of owning a complex systems component end to end, from design through production, and driving technical decisions across teams
  • Understanding of LLM inference workloads and what they demand of a runtime
  • Comfort building software for hardware that is still evolving, from simulation through bring-up

BONUS POINTS

  • Experience with open runtime designs such as IREE's HAL, or with CUDA or ROCm runtime internals
  • Experience with in-memory compute or other non-von-Neumann hardware
  • Experience integrating a runtime with an inference server such as vLLM
  • Contributions to open-source runtime or systems infrastructure

Equal Employment Opportunity Statement

Normal Computing is an Equal Opportunity Employer. We celebrate diversity and are committed to creating an inclusive environment for all employees. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, veteran status, or any other legally protected status.

Accessibility Accommodations

Normal Computing is committed to providing reasonable accommodations to individuals with disabilities. If you need assistance or an accommodation due to a disability, please let us know at accommodations@normalcomputing.com.

Privacy Notice

By submitting your application, you agree that Normal Computing may collect, use, and store your personal information for employment-related purposes in accordance with our Applicant Privacy Notice https://normalcomputing.com/applicant-privacy-notice.

Source: Normalcomputing careers (Ashby)

Similar roles

More roles at Normalcomputing