[Hiring] Senior Machine Learning Systems Engineer @Atlassian

Role Description

As a Senior ML System Engineer on the AI & ML Platform’s Inference team, you will design and optimize large-scale model serving systems end-to-end. You will own everything from distributed infrastructure (global KV cache, continuous batching, load balancing, auto-scaling) to deep low-level optimizations (GPU kernels, quantization, speculative decoding).

• Architect and implement scalable distributed infrastructure for model serving (load balancing, auto-scaling, batch scheduling, global KV cache).
• Optimize latency and throughput of model inference under real production workloads.
• Build reliable, high-concurrency serving systems that serve billions of requests reliably.
• Benchmark, fine-tune, and accelerate inference engines.
• Create robust CI/CD infrastructure for seamless model deployment and inference engine updates.
• Partner with senior ML engineers to fine‑tune open-source LLMs and deploy.

Qualifications

• 5+ years of software engineering experience, 2+ years of system performance optimization experience.
• Deep low-level systems programming (C/C++ or Rust).
• Experience with large-scale, high-concurrent production serving.
• Experience with GPU inference engines (vLLM, SGLang, Triton, TensorRT-LLM, etc.).
• Strong background in system optimizations: batching, caching, load balancing, parallelism.

Requirements

• Low-level inference optimizations: GPU kernels.
• Algorithmic inference optimizations: quantization, speculative decoding, distillation.
• Experience with testing, benchmarking, and reliability of inference services.
• Experience designing and implementing CI/CD infrastructure for inference.

Benefits

• Health and wellbeing resources.
• Paid volunteer days.
• Wide range of perks and benefits designed to support you and your family.