[Hiring] Engineering Manager, Online Serving and Inference, AI Platform @Netflix

Role Description

At Netflix, our mission is to entertain the world. Together, we are writing the next episode - pushing the boundaries of storytelling, global fandom and making the unimaginable a reality.

The Online Serving and Inference team builds and maintains the infrastructure that enables ML practitioners to run their large-scale online serving and inference workflows to deploy AI models for Netflix member scale ecosystems. The team is responsible for the common framework to develop and deploy AI/ML models in Netflix infrastructure to power the AI applications and features.

We are looking for an experienced AI/ML infrastructure engineering leader to lead the development of our next-generation serving and inference platform!

• Partner with Applied Research and backend application teams to understand their needs and gather and iterate on the platform's requirements.
• Drive the architecture, design, build vs. buy evaluation, and execution of the online serving and inference platform.
• Build a roadmap focused on incremental delivery. Define success metrics, align on migration goals, and drive the platform's adoption.
• Communicate progress to stakeholders, customers, and senior leadership.
• Maintain existing platform offerings, balancing immediate needs in current systems while prioritizing the development of the next-gen platform.
• Hire and grow diverse, highly talented engineers while maintaining and fostering an inclusive team culture.

Qualifications

• 10+ years of software engineering experience and 3+ years of management experience.
• Experience leading teams responsible for building high-traffic distributed systems and ML infrastructure.
• Experience with containerization and orchestration technologies to support data preparation, processing, and inference for large-scale AI/ML models, such as Kubernetes.
• Industry experience working with GPU based mode inference, and serving optimization (batching, latency, cost per request) in large scale computing environments.
• Modern Foundation Model, such as LLM, serving and inference stacks such as vLLM, Nvidia Triton, TensorRT LLM, or SGLang.
• Strong technical acumen and can act as a credible technical advisor to the team.
• A passion for translating the needs of ML practitioners into infrastructure offerings with an eye toward automated and self-serve capabilities.
• Strong communication and collaboration skills and ability to build strong relationships with internal customers and external partners.
• A demonstrated ability to develop, drive, and execute a technical vision and roadmap.
• A track record of attracting top talent and leading and growing high-performance, diverse, and highly talented senior and staff level engineers.
• Experience managing a hybrid team with partners and team members distributed across (US) geographies & time zones.

Requirements

• Experience with GPU based mode inference, and serving optimization (batching, latency, cost per request) in large scale computing environments.
• Modern Foundation Model, such as LLM, serving and inference stacks such as vLLM, Nvidia Triton, TensorRT LLM, or SGLang.

Benefits

• Comprehensive Health Plans.
• Mental Health support.
• 401(k) Retirement Plan with employer match.
• Stock Option Program.
• Disability Programs.
• Health Savings and Flexible Spending Accounts.
• Family-forming benefits.
• Life and Serious Injury Benefits.
• Paid leave of absence programs.
• Full-time hourly employees accrue 35 days annually for paid time off.
• Full-time salaried employees are immediately entitled to flexible time off.