[Hiring] Senior Performance Engineer - LLM Inference Optimization @Active Impact Investments

Role Description

Are you driven by the toughest challenges in AI? We're seeking a Senior Performance Engineer to join our founding team and redefine LLM inference. In this role, you'll architect high-impact solutions across the entire stack—from optimizing kernels, shaping the serving engine, improving the scheduling/orchestration, and advancing open-source inference engines. If you're ready to push the boundaries of AI performance, we want to hear from you.

What you'll do

• Design, build, and optimize high-performance inference systems for LLMs and multimodal models, from single-GPU experiments to multi-GPU, multi-node deployments.
• Develop and tune custom kernels and runtime paths (CUDA / TensorRT / Triton / compiler-based optimizations) to push GPU utilization and memory efficiency toward theoretical limits.
• Improve and extend serving engine (e.g., vLLM, TensorRT-LLM, SGLang, or custom runtimes) including batching, KV-cache management, scheduling, and routing policies for heterogeneous workloads (sync, async, batch, streaming).
• Design and operate distributed inference systems: autoscaling policies, load balancing, capacity management, and failure handling under strict latency and reliability SLOs.
• Instrument, profile, and analyze the entire inference lifecycle (from request to GPU to network and storage) to identify bottlenecks and drive systematic performance improvements.
• Collaborate with hardware partners and internal infra teams to co-design software/hardware strategies that improve cost and energy efficiency (e.g., hardware SKUs, topology-aware placement, mixed precision, quantization).
• Explore and implement advanced inference techniques such as speculative decoding, paged KV-cache, tensor/parallellism, LoRA/multi-LoRA serving, mixture-of-experts, and offloading for memory-constrained environments.
• Work closely with research and product teams to translate abstract model and product requirements into concrete constraints and system designs, balancing latency, throughput, quality, and cost.
• Contribute to internal tooling and, when appropriate, open-source projects around inference engines, benchmarking suites, and observability.

Qualifications

• Deep experience building or operating large-scale inference or training systems in production (LLMs preferred, but other large-scale deep learning systems also relevant).
• Understand GPU architectures and performance characteristics (compute vs memory vs bandwidth, kernel launch overheads...).
• Comfortable with the modern inference ecosystem: vLLM, TensorRT / TensorRT-LLM, Triton, SGLang, or similar.
• Strong understanding in systems and distributed-systems fundamentals (data structures, concurrency, scheduling, backpressure, observability, fault tolerance).
• Enjoy debugging performance issues across layers: from kernel traces and CUDA profiles to queueing behavior in the serving layer and autoscaler dynamics.
• Comfortable in a research-adjacent environment: reading papers, prototyping new algorithms, exploring emerging hardware, and turning prototypes into robust systems.
• Can own problems end-to-end, from initial exploration and measurement to deployed systems and long-term operational health.

Requirements

• Contributions to inference frameworks or libraries (e.g., vLLM, SGLang, TensorRT-LLM, FasterTransformer, custom CUDA/Triton kernels).
• Experience with K8s-based infrastructure (e.g., operators, custom controllers, autoscalers) and modern observability stacks.
• Work on cost or energy-aware scheduling, capacity planning, or optimization for large fleets of accelerators.
• Experience with post-training pipelines (RL, preference optimization, evals) and their interaction with inference infrastructure.

Benefits

• Mission that compounds. Power is the binding constraint for AI deployment this decade. You'll build the layer that makes every joule count - and every token more useful.
• World-class science bench. Work alongside two Canada CIFAR AI Chairs and core Mila researchers: Pierre-Luc Bacon (UdeM, scientific co-founder) and Christophe Dubach (McGill, scientific advisor). Full support of Mila, Canada's leading AI research institute, with access to its talent pipeline and research bench.
• Real hardware from day one. H100, H200, AMD MI-series, and design-partner engagements already in motion.
• Resources to match the ambition. Seed-funded by strong operators and institutional investors. We have the resources to hire the best, give them the compute they need, and run the long experiments that matter. Founding-engineer equity, direct ownership, no bureaucracy - your code reaches production in weeks, not quarters.
• Remote-friendly. Montreal preferred for regular in-person work with the founding team.

Company Description

CarbonForge solves challenges in AI Inference Optimization, optimizing the full stack from hardware and kernels to serving engines and APIs to maximize tokens-per-watt and tokens-per-dollar. Founded by AI researchers and entrepreneurs from Mila, we deliver low-latency, high-throughput inference that meets the most demanding power and cost constraints.