[Hiring] Member of Technical Staff, Inference @Inferact

Role Description

We're looking for an inference runtime engineer to push the boundaries of what's possible in LLM and diffusion model serving. Models grow larger. Architectures shift: mixture-of-experts, multimodal, agentic. Every breakthrough demands innovations on the inference engine itself. You'll work at the core of vLLM, optimizing how models execute across diverse hardware and architectures. Your work will directly impact how the world runs AI inference.

Qualifications

• Bachelor's degree or equivalent experience in computer science, engineering, or similar
• Deep understanding of transformer architectures and their variants
• Strong programming skills in Python with experience in PyTorch internals
• Experience with LLM inference systems (vLLM, TensorRT-LLM, SGLang, TGI)
• Ability to read and implement model architectures and inference techniques from research papers
• Demonstrate the ability to contribute performant and maintainable code and debug in complex ML codebases

Requirements

• Deep understanding of KV-cache memory management, prefix caching, and hybrid model serving
• Familiarity with RL frameworks and algorithms for LLMs
• Experience with multimodal inference (audio/image/video/text)
• Contributions to open-source ML or system infrastructure projects

Benefits

• Generous health, dental, and vision benefits
• 401(k) company match

Logistics

• Location: This role is based in San Francisco, California. Will consider remote in the US for exceptional candidates
• Compensation: Depending on background, skills, and experience, the expected annual salary range for this position is $200,000 - $400,000 USD + equity
• Visa sponsorship: We sponsor visas on a case-by-case basis