[Hiring] Technical Lead - AI Inferences @WEKA

Role Description

We are seeking a Technical Lead - AI Inferences to spearhead our AI Inference team. In this role, you will bridge the gap between complex research and production-grade engineering. You will lead a tight-knit squad of 3 developers while remaining "hands-on-keyboard," architecting high-performance systems that optimize Large Language Model (LLM) serving.

Responsibilities

• Technical Leadership: Architect and oversee the deployment of high-throughput, low-latency LLM inference pipelines.
• Team Management: Mentor and lead a small team of developers, conducting code reviews, sprint planning, and technical career coaching.
• Inference Optimization: Implement and evaluate state-of-the-art KV cache management solutions, including LMCache, and explore alternatives to minimize redundant computation.
• Framework Mastery: Deeply integrate and optimize serving engines such as vLLM, LLM-d, and NIXL to maximize hardware utilization.
• R&D: Stay at the forefront of the "Inference-as-a-Service" domain, benchmarking new tools and deciding when to pivot the stack.

Qualifications

• Proven experience with KV cache reuse, speculative decoding, and continuous batching.
• Deep familiarity with vLLM, LMCache, and NIXL. Understanding the trade-offs between centralized vs. distributed caching.
• Expertise in Python, C++, or Rust, with a strong grasp of CUDA and GPU memory management.
• Experience with Kubernetes (K8s) for scaling GPU workloads and optimizing cold-start times.

Benefits

• Medical, Dental, Vision, Life insurance.
• 401(K) plan.
• Flexible Time Off (FTO).
• Sick time and leave of absence as per the FMLA and other relevant leave laws.