[Hiring] Senior Applied ML Engineer @Kimchi

Role Description

Every request gets a decision: which model serves it. This is an applied-ML seat, not an infrastructure one. Platform engineers own the proxy, the streaming path, and the fleet. You own the model inside them: the training data, the objective, the evaluation, and the iteration loop that ships a better router every cycle.

The problem:

• Routing is a prediction problem wearing a product's clothes.
• For every request, you predict the quality each candidate model would deliver, what it would cost, and how fast it would decode.
• You must be right enough, in aggregate, that the savings are real and the quality never dips below frontier.
• The data is messy: rollout outcomes from benchmark harnesses, per-request records from the live fleet, override signals, and token streams whose speed depends on the serving concurrency at the moment.
• Speed itself is a correctness variable: on deadline-bound agentic work, a model that decodes three times slower doesn't do the same work slower, it does less work.
• A router change ships only after production-parity evaluation with error bars, then gets held to its numbers in live traffic.

Stack:

• Python
• PyTorch
• SQL at scale (ClickHouse)
• OpenAI- and Anthropic-compatible LLM APIs
• Benchmark and rollout harnesses (RouterBench-class)
• GitLab CI

Qualifications

• 5+ years in applied ML or ML research.
• Experience in training, evaluating, and shipping models that make real-time decisions (ranking, recommendation, bandits, pricing, or LLM routing).
• Strong Python and SQL at scale skills.
• Understanding of model evaluation and benchmarks.
• Knowledge of serving reality (TTFT, decode speed, batching, prefix caching, concurrency).
• Ability to instrument before optimizing.
• Excitement about a wide mandate.

Requirements

• Experience with LLM routing specifically: RouterBench-class evals, cost/quality/latency Pareto fronts.
• Familiarity with calibration, abstention, or conformal methods in production decision systems.
• Experience running online A/B loops with drift monitoring on a model that touches real money.

Responsibilities

• Own the routing model end to end: features, objective, training, calibration, and release cadence.
• Build the evaluation that gates every change: production-parity benchmarks, router-vs-router comparisons, holdouts by task family, error bars on pass rates.
• Turn fleet data into training signal: per-request outcomes from production, benchmark rollouts, and override feedback.
• Model the objective the user actually experiences: task success × token cost × wall-clock speed under the real concurrency regime.
• Run the online loop: shadow, A/B, full rollout, drift monitoring, retrain.
• Push the frontier where it pays: calibrated decisions, abstention under uncertainty, speed-aware routing.
• Partner with platform engineers on instrumentation: features and labels recorded per request.

Benefits

• Competitive salary (depending on the level of experience).
• Flexible, remote-first global environment.
• Collaboration with a global team of cloud experts and innovators.
• Equity options.
• Quick feedback with a fast-paced workflow.
• 10% of work time on personal projects or self-improvement.
• Learning budget for professional and personal development.
• Annual hackathon to spark new ideas and strengthen team bonds.
• Team-building budget and company events.
• Equipment budget to ensure you have everything you need.
• Extra days off to maintain a healthy work-life balance.

Hiring Process

• Screening call with Recruiter
• Hiring Manager interview
• Technical interview (system design)
• Live coding
• Culture Check interview with an executive

As part of our standard hiring process, we would like to inform you that a background check may be conducted at the final stage of recruitment through our third-party provider, Checkr.

Please note that Cast AI does not provide any form of visa sponsorship/work permit.