Software Engineer 5, AI Observability @Netflix
Artificial Intelligence
Salary usd 500,000 - 8..
Remote Location
πŸ‡ΊπŸ‡Έ USA Only
Employment Type full-time
Posted YDay

[Hiring] Software Engineer 5, AI Observability @Netflix

YDay - Netflix is hiring a remote Software Engineer 5, AI Observability. πŸ’Έ Salary: usd 500,000 - 860,000 per year πŸ“Location: USA

Role Description

At Netflix, our mission is to entertain the world. Together, we are writing the next episode - pushing the boundaries of storytelling, global fandom and making the unimaginable a reality. We are a dream team obsessed with the uncomfortable excitement of discovering what happens when you merge creativity, intuition and cutting-edge technology. Come be a part of what’s next.

The AI Observability team makes AI, ML, and Agentic systems transparent, reliable, and production-ready at scale. We build end-to-end observability for ML and GenAI workloads, capturing model inputs, features, predictions, outcomes, and behavior across online and batch systems. Our platform enables teams to monitor model performance, data quality, drift, latency, and failures, turning the ML system from a black box into an explainable, debuggable system. We provide developer-friendly libraries, dashboards, and alerts so teams can debug issues, respond to incidents, and ship AI-powered products with confidence.

We're looking for a hands-on senior engineer to build the frameworks behind Netflix's AI Observability platform, model performance, evaluation, and vendor integration surfaces. You will design reusable infrastructure that enables ML/AI practitioners across domains to monitor model quality in production, evaluate LLM and agentic systems, and adopt vendor tooling through a consistent, self-serve platform. AIP owns the generic, reusable infrastructure; domain teams own their domain-specific evals and remediation. You will partner closely with engineering, product, machine learning, and data teams to turn their needs into reusable platform capabilities. To succeed, you will bring a strong background in AI or ML infrastructure and a passion for building scalable, robust systems.

In this role, you will:

  • Build the observability framework and platform capabilities that give ML and GenAI systems metrics, logs, and distributed traces across online inference, batch scoring, feature pipelines, and agent orchestration.
  • Build the primitives that let teams monitor model performance (accuracy, calibration, error rates), data quality, drift, and degradation on their own systems.
  • Build and extend evaluation frameworks for LLM and agentic systems that support response quality, grounding and hallucination, task success, tool-use and trajectory correctness, and LLM-as-a-judge and human-in-the-loop scoring.
  • Lead build-vs-buy evaluations for observability and eval tooling, and own the SDKs, connectors, and APIs that integrate vendor platforms into a consistent, well-supported interface.
  • Build reusable libraries, SDKs, and templates that make observability and evaluation the default for new systems ("observability-by-default").
  • Provide the dashboarding, alerting, and SLO/SLI building blocks (plus sensible out-of-the-box templates) that teams use to track model performance, latency, cost, and reliability.

Qualifications

  • Experience in software, AI/ML, or platform engineering, with hands-on time in production observability, monitoring, or ML/LLM evaluation.
  • Proven track record of designing standards, libraries, SDKs, or frameworks that other teams build on and adopt.
  • Strong coding skills in Python and at least one of Java, Go, or Scala, with experience building production services.
  • Practical experience operating ML models in production (online serving and/or batch), including drift, model performance, and data quality.
  • Hands-on experience with modern observability stacks (e.g., Prometheus/Grafana, Datadog, OpenTelemetry, ELK/OpenSearch, Jaeger/Tempo, or similar).
  • Solid understanding of distributed systems, microservices, and at least one major cloud platform (AWS, GCP, or Azure).
  • Ability to work cross-functionally with ML, data, infra, and product teams, and to communicate clearly about system behavior, quality, and risk.
  • Experience with vendor integration and VPC deployment.
  • An AI-Native Engineering Mindset who uses AI tools as a core part of their own workflow.

Requirements

  • Hands-on experience with ML/LLM observability and evaluation tools (e.g., Arize, Braintrust, LangFuse, Weights & Biases, Galileo, Vertex AI Model Monitoring, SageMaker Model Monitor).
  • Experience building or shipping LLM/GenAI applications and evaluating them: prompt/result logging, evaluation metrics, LLM-as-a-judge, and human-in-the-loop review.
  • Experience evaluating agentic systems: tool use, multi-step reasoning, and trajectory/task-success measurement.

Benefits

  • Comprehensive Health Plans.
  • Mental Health support.
  • 401(k) Retirement Plan with employer match.
  • Stock Option Program.
  • Disability Programs.
  • Health Savings and Flexible Spending Accounts.
  • Family-forming benefits.
  • Life and Serious Injury Benefits.
  • Paid leave of absence programs.
  • Full-time hourly employees accrue 35 days annually for paid time off.
  • Full-time salaried employees are immediately entitled to flexible time off.
Before You Apply
️
πŸ‡ΊπŸ‡Έ Be aware of the location restriction for this remote position: USA Only
β€Ό Beware of scams! When applying for jobs, you should NEVER have to pay anything. Learn more.
Software Engineer 5, AI Observability @Netflix
Artificial Intelligence
Salary usd 500,000 - 8..
Remote Location
πŸ‡ΊπŸ‡Έ USA Only
Employment Type full-time
Posted YDay
Apply for this position
Did not apply βœ“
Applied βœ“
Sent Follow-Up βœ“
Interview Scheduled βœ“
Interview Completed βœ“
Offer Accepted βœ“
Offer Declined βœ“
Application Denied βœ“
Unlock 140,000+ Remote Jobs
️
πŸ‡ΊπŸ‡Έ Be aware of the location restriction for this remote position: USA Only
β€Ό Beware of scams! When applying for jobs, you should NEVER have to pay anything. Learn more.
Apply for this position
Did not apply βœ“
Applied βœ“
Sent Follow-Up βœ“
Interview Scheduled βœ“
Interview Completed βœ“
Offer Accepted βœ“
Offer Declined βœ“
Application Denied βœ“
Unlock 140,000+ Remote Jobs
Γ—
Apply to the best remote jobs
before everyone else

Access 140,000+ vetted remote jobs and get daily alerts.

4.9 β˜…β˜…β˜…β˜…β˜… from 500+ reviews

⚑ 141,313+ remote jobs, refreshed hourly

πŸ”” Real-time alerts: Apply first, direct to employer

πŸ›‘οΈ Vetted companies, no scams, true remote only

Unlock All Jobs Now

Maybe later