[Hiring] Senior Applied ML Engineer @Cast AI

Role Description

Every request gets a decision: which model serves it. This is an applied-ML seat, not an infrastructure one. Platform engineers own the proxy, the streaming path, and the fleet. You own the model inside them: the training data, the objective, the evaluation, and the iteration loop that ships a better router every cycle.

The problem:

• Routing is a prediction problem wearing a product's clothes.
• For every request, you predict the quality each candidate model would deliver, what it would cost, and how fast it would decode.
• You have to be right enough, in aggregate, that the savings are real and the quality never dips below frontier.
• The data is messy: rollout outcomes from benchmark harnesses, per-request records from the live fleet, override signals, and token streams whose speed depends on the serving concurrency at the moment.
• Speed itself is a correctness variable: on deadline-bound agentic work, a model that decodes three times slower doesn't do the same work slower, it does less work.
• Naive cost-per-token objectives quietly optimize for the wrong thing.
• The bar for evidence is high: a router change ships only after production-parity evaluation with error bars, then gets held to its numbers in live traffic.

Stack:

• Python
• PyTorch
• SQL at scale (ClickHouse)
• OpenAI- and Anthropic-compatible LLM APIs
• Benchmark and rollout harnesses (RouterBench-class)
• GitLab CI

Qualifications

• You've spent 5+ years in applied ML or ML research, and you've trained, evaluated, and shipped models that make real-time decisions (ranking, recommendation, bandits, pricing, or LLM routing) and you can tell that story with receipts.
• Strong Python and SQL at scale: you can pull, clean, and interrogate millions of request-level records without waiting on anyone.
• You know the difference between a model that got better and an eval that got easier.
• You have real opinions about what LLM benchmarks measure, agentic harnesses included.
• You know enough serving reality (TTFT, decode speed, batching, prefix caching, concurrency) to model the objective honestly.
• You instrument before you optimize, error bars and all.
• A wide mandate excites you more than it unsettles you.

Requirements

• You've worked on LLM routing specifically: RouterBench-class evals, cost/quality/latency Pareto fronts.
• Calibration, abstention, or conformal methods in production decision systems.
• You've run online A/B loops with drift monitoring on a model that touches real money.

Responsibilities

• Own the routing model end to end: features, objective, training, calibration, and release cadence. Iterate it like a product, not a research artifact.
• Build the evaluation that gates every change: production-parity benchmarks, router-vs-router comparisons, holdouts by task family, error bars on pass rates.
• Nothing ships on vibes.
• Turn fleet data into training signal: per-request outcomes from production, benchmark rollouts, and override feedback, with failure categorization (infra vs task) so poisoned labels never reach the policy.
• Model the objective the user actually experiences: task success × token cost × wall-clock speed under the real concurrency regime, with cached-input economics in the picture.
• Run the online loop: shadow, A/B, full rollout, drift monitoring, retrain. When production disagrees with your offline eval, you find out why before the next ship.
• Push the frontier where it pays: calibrated decisions, abstention under uncertainty, speed-aware routing. We experiment on what a router should be, not just which model it picks today.
• Partner with platform engineers on instrumentation: the features and labels recorded per request (decode speed first-class among them) and the experiments the data path must support.

Benefits

• Competitive salary (depending on the level of experience).
• Enjoy a flexible, remote-first global environment.
• Collaborate with a global team of cloud experts and innovators, passionate about pushing the boundaries of Kubernetes technology.
• Equity options.
• Get quick feedback with a fast-paced workflow. Most feature projects are completed in 1 to 4 weeks.
• Spend 10% of your work time on personal projects or self-improvement.
• Learning budget for professional and personal development - including access to international conferences and courses that elevate your skills.
• Annual hackathon to spark new ideas and strengthen team bonds.
• Team-building budget and company events to connect with your colleagues.
• Equipment budget to ensure you have everything you need.
• Extra days off to help maintain a healthy work-life balance.

Hiring Process

• Screening call with Recruiter
• Hiring Manager interview
• Technical interview (system design)
• Live coding
• Culture Check interview with an executive

As part of our standard hiring process, we would like to inform you that a background check may be conducted at the final stage of recruitment through our third-party provider, Checkr.

Please note that Cast AI does not provide any form of visa sponsorship/work permit.