[Hiring] Staff/ Principal Software Engineer @Netflix

Role Description

Netflix is on the journey to rebuild the AI/ML model serving platform and infrastructure that turns trained models into served predictions across real-time, async, and agentic workloads, at a scale few companies operate at. Supporting many production clusters, from classical to foundation models, spanning latencies from milliseconds rankers to LLM calls, and powering everything members see from homepage ranking to search, messaging, and internal AI productivity tools.

We are re-building a new model serving platform from the ground up, designed from first principles to meet industry standards. It unifies our in-house inference infrastructure, third-party model hosting, and external LLM providers under a single serving, routing, and experimentation layer. Teams get one consistent way to deploy, route, and A/B test any model, regardless of where it runs.

This role sits at the center of Netflix’s GenAI and Foundation Model serving strategy, with real R&D scope on frontier AI infrastructure. You will tackle the efficiency problems that most directly drive cost per inference and capacity:

• Disaggregated serving
• Intelligent routing across models and providers
• Speculative decoding
• Prompt caching
• Multi-GPU inference

You will lead the technology behind some of Netflix’s largest bets, embedding AI capabilities into member-facing products at global scale. You will also build durable, resumable execution for long-running agentic workloads. Online and asynchronous/Near-Real-Time inference is projected to grow fast in 2028, so getting this foundation right now is what keeps costs sustainable and reliability high as demand scales.

This is a rare very senior level opportunity to set and own the technical direction for a platform at the intersection of massive production scale, aggressive cost/utilization engineering, and genuinely unsolved problems in GenAI serving by leading the technical architecture and directions for a growing engineering organization.

Qualifications

• Have a BS/MS/PhD in Computer Science, Applied Math, Engineering, or a related field.
• Relevant experience in the AI/ML Infrastructure and distributed systems for 12+ years.
• Influence without authority to guide architectural decisions and alignment across multi-teams through trust, data, and demonstrated expertise.
• Shape long-term technical roadmaps (typically looking 2 to 3 years ahead) and ensure scalability, maintainability, and business alignment.
• Cross-Functional Partnership: Work closely with product managers, operations, and executive stakeholders to balance technical trade-offs with business priorities.
• Clear Communication skill to translate dense technical concepts for non-technical stakeholders and produce clear documentation, architecture decision records (ADRs), and proposals.
• Experience with high-traffic distributed services and infrastructure for AI/ML model inference, focusing on high availability, security, and performance.
• Experience building scalable model-serving solutions for Foundation Models that power generative and predictive AI applications with practical skills in reducing latency and costs.
• Familiar with deploying AI/ML models using platforms and solutions such as NVIDIA Triton Inference Server, vLLM, or SGLang, and scaling them via Kubernetes.
• Experience working with public cloud infrastructure providers such as AWS, Azure, or GCP.

Requirements

• Open-source contributions to projects in the AI/ML, distributed systems, or infrastructure space.
• Hands-on experience with building AI models and exploring optimization techniques for AI models, such as quantization, model compression, or model pruning.
• Proficient in leveraging AI coding tools in software development.
• Experience working with model gateway and intelligent routing for choosing the right AI models based on the context of the requests.
• Exposure to deploying multi-adapters model serving, disaggregated prefill/decode, or large-scale KV cache management.
• Working with evaluation and benchmarking tools for LLMs and AI Agents.
• Knowledge of GPU topology, NVIDIA CUDA programming, and machine learning compilers.

Benefits

• Comprehensive Health Plans
• Mental Health support
• 401(k) Retirement Plan with employer match
• Stock Option Program
• Disability Programs
• Health Savings and Flexible Spending Accounts
• Family-forming benefits
• Life and Serious Injury Benefits
• Paid leave of absence programs
• Full-time hourly employees accrue 35 days annually for paid time off
• Full-time salaried employees are immediately entitled to flexible time off