[Hiring] Research Engineer @Gauntlet

Role Description

Gauntlet is looking for an AI Research Engineer to help improve how we build, evaluate and use AI agents across the business. We already have agents running in production, including Aria, our in-house generalist agent. They're used across research, engineering and business workflows, and we've learned a lot from seeing how they perform in practice. Some tasks work extremely well; others produce inconsistent results, and changes to models or tooling can introduce unexpected problems. Today, much of that feedback is still subjective. We want to build a more systematic way of understanding what works, what doesn't and why.

A major part of this role will be building the evaluation frameworks that allow us to measure agent performance on real tasks. You'll work with our engineers, quantitative researchers and other teams to understand where agents struggle, turn those observations into measurable experiments, and use the results to improve the systems we're building.

You'll also contribute directly to our existing agents and internal tooling. Initially, the work will be more engineering-focused as you get familiar with our systems, shipping improvements and integrations that teams can use. Over time, you'll take on more open-ended research, helping us develop better approaches to evaluation, agent behaviour and infrastructure.

We're also thinking beyond individual agents. One of our longer-term goals is to build a shared system of knowledge and reusable skills that our agents can draw from and improve over time. You'll play an important role in developing that infrastructure and how we evaluate its effectiveness.

This is an applied research role with a meaningful engineering component. We're looking for someone who enjoys investigating difficult problems, can design good experiments and is comfortable building and shipping the solutions that come out of them.

Qualifications

• Hands-on experience building, evaluating or improving LLM models or agents, whether through professional work, applied research, open-source contributions or substantial personal projects.
• Strong Python skills and good software engineering fundamentals. You're comfortable writing production-quality code and understand the importance of testing, maintainability and reliability.
• Experience with applied research, machine learning or data science, including designing experiments, working with datasets and drawing conclusions from results.
• A good understanding of how LLM agents work beyond prompting, including tool use, harnesses, context management and common failure modes.
• The ability to take an open-ended problem, break it down into testable questions and work independently towards a solution.
• Experience building tools or systems that other people use, with an understanding of how to evaluate their effectiveness beyond technical benchmarks.
• Comfort working across research and engineering, collaborating with different teams and adapting as priorities change.
• Genuine interest in AI agents and how they can be made more capable, reliable and useful.

Requirements

• Experience building LLM evaluation pipelines, benchmarks, reinforcement learning environments or model testing infrastructure.
• Contributions to open-source agent frameworks, harnesses, evaluation datasets or benchmarks.
• Experience with agent frameworks such as Hermes, OpenClaw or similar tools.
• Experience in applied ML research, model training, fine-tuning or reinforcement learning.
• An advanced degree or postgraduate research experience in computer science, mathematics, statistics, machine learning or a related field.
• Experience in financial markets, quantitative research, DeFi or risk modelling.
• Familiarity with AI security considerations, including access controls, prompt injection and data protection.

Benefits

• Remote first — work from anywhere in the United States
• Regular in-person company retreats and cross-country "office visit" perk
• 100% paid medical, dental, and vision premiums for employees
• $1,000 WFH stipend
• Monthly reimbursement for home internet, phone, and cellular data
• Unlimited vacation
• 100% paid parental leave of 12 weeks
• Fertility benefits
• Opportunity for incentive compensation