[Hiring] Senior Staff Engineer - Infrastructure Systems @Runware

Role Description

As Senior Staff Engineer, Infrastructure Systems, you will own the GPU software stack on every Runware node: the firmware, the kernel and NVIDIA driver, the CUDA runtime and libraries, the container runtime, and the way Kubernetes and our inference services see and share a GPU. When a GPU misbehaves anywhere in the fleet, you will find out why and write the software that keeps it from happening again.

This is a senior staff-level individual contributor role for someone who knows GPUs from the driver up and still writes code every day. Success means that every GPU in every pod runs a known, tested combination of firmware, driver and CUDA, that upgrades roll across the fleet without failed inferences, that GPU faults are detected, classified and remediated automatically, and that a new GPU generation goes from delivery to serving traffic in days.

What you'll do

• Own the GPU software baseline: versioned, reproducible combinations of kernel, NVIDIA driver, firmware, CUDA runtime and libraries (cuDNN, cuBLAS, NCCL, TensorRT) and container toolkit, tested before they roll and rolled across the fleet without downtime.
• Qualify every driver and CUDA release: keep the compatibility matrix across GPU generations, frameworks and model families, and catch regressions in correctness, performance and memory use before they reach production.
• Take GPU failures to root cause: Xid errors, CUDA "unknown error", cuBLAS handle failures, out-of-memory and fragmentation, GPUs that fall off the bus.
• Build GPU acceptance and diagnostics: the burn-in, DCGM diagnostics, memory and interconnect tests and real-workload tests that gate every node and rack before it serves traffic.
• Make GPU health visible and self-healing: pull telemetry from DCGM, the driver, the kernel log and the BMC into one trusted view, classify faults, and automate the drain, reset and replacement paths.
• Own how GPUs are exposed to workloads: the container runtime, device plugins and the GPU Operator, MIG and sharing policies, and isolation between tenants, on bare metal and on Kubernetes.
• Bring up new GPU generations and new pods with the hardware and node operations teams: first driver and firmware, NVLink and NVSwitch validation, and the automation that makes the next bring-up take days.
• Characterize hardware with real workloads: benchmark image, video, audio and LLM inference on each GPU generation and software stack.
• Work with NVIDIA and upstream projects: file driver and firmware bugs with reproducible cases and see them through, track what each release changes, and contribute fixes to the open-source tools we depend on.
• Use AI to accelerate the work: agents for log and Xid triage, test generation, release qualification and runbook automation.

Qualifications

• 12+ years building and operating systems software, with several of those on GPU software running in production at scale.
• Deep, hands-on knowledge of the NVIDIA software stack: the kernel driver and its modules, the CUDA runtime and libraries, NVML, DCGM and the Container Toolkit.
• A record of taking GPU failures to root cause across hardware, firmware, driver, runtime and application.
• Strong Linux internals: the kernel, device drivers and modules, PCIe, NUMA and IOMMU, cgroups and containers.
• You write code every day: fluent in Python and in at least one systems language (C, C++, Rust or Go).
• Experience running GPUs under Kubernetes and on bare metal: device plugins, the GPU Operator, MIG, image builds and provisioning at scale.
• Working knowledge of how inference frameworks use a GPU (PyTorch, TensorRT, ONNX Runtime).
• Clear technical writing: your design documents, postmortems and ticket updates can be acted on by someone who was not in the room.
• You use AI to accelerate your own output every day.
• Comfortable owning outcomes in a fast-moving, ambiguous environment.

Requirements

• Time spent building GPU drivers, CUDA libraries or GPU system software at a GPU vendor, a cloud provider or an HPC centre.
• Experience with Blackwell-generation hardware (B200, GB200/GB300 NVL72), NVLink and NVSwitch topologies, and liquid-cooled racks.
• Multi-GPU and multi-node execution: NCCL, GPUDirect, and RDMA over RoCE or InfiniBand.
• Firmware qualification across BIOS, BMC, NIC and GPU, and acceptance testing with OEMs and ODMs.
• Inference serving internals: PyTorch, TensorRT-LLM, vLLM, SGLang or custom diffusion and video pipelines.
• Experience with non-NVIDIA accelerators (AMD or custom silicon) and their software stacks.
• An open-source project you wrote or maintain that people run in production.
• Multi-tenant isolation and confidential computing on shared GPU hardware.
• Experience running a 24x7 service with published service levels.

Benefits

• Generous paid time off: vacation, sick days, public holidays.
• Meaningful stock options: share in the upside you create.
• Flexible hours: own your schedule outside core collaboration blocks.
• Family leave: paid maternity, paternity, and caregiver time.
• Company retreats: twice-yearly gatherings in inspiring locations.