[Hiring] Software Engineer, Cybersecurity Benchmarks @Epoch AI

Role Description

Epoch AI is looking for a Software Engineer, Cybersecurity Benchmarks who will help us measure how capable frontier AI models are at cybersecurity, and keep those measurements safe to run. The role will involve building and owning cybersecurity benchmarks, securing our evaluation environments against AI escape, and using threat modelling to shape our evaluation strategy.

We are looking for a Research Engineer to help us build trustworthy, public evaluations of AI cybersecurity capabilities as part of our AI Benchmarking Hub. You will work closely with the benchmarking team, researchers, analysts, and other engineers to design benchmarks, run them on our infrastructure, and make sure the results are accurate and feed into our research.

Your day-to-day activities will include:

• Building and maintaining cybersecurity benchmarks (primarily in the Inspect library)
• Hardening the sandboxes they run in
• Keeping our benchmarks running to evaluate new model releases quickly

An ideal candidate for this role might have a background in red teaming and sandboxing software, alongside strong software engineering experience and hands-on familiarity with AI coding harnesses. Your work will be key to giving researchers, developers, and policymakers an independent, rigorous picture of what AI can do in cybersecurity.

Qualifications

• Strong software engineering experience, including several years building and maintaining complex systems and regularly contributing robust, maintainable code
• Experience sandboxing software against highly capable adversaries
• Red team experience ethically hacking software
• Familiarity with agentic AI use and coding harnesses
• Ideas and creativity: you can generate your own ideas for new benchmarks and experiments
• Motivation from Epoch AI’s mission to provide rigorous, independent insight into key trends in AI, including delivering public, trustworthy evaluations of AI capabilities

Requirements

• Experience benchmarking AIs and running evaluations
• Experience setting up CTF infrastructure
• Experience using modern AI for red teaming or ethically hacking
• Knowledge of the inner workings of AI coding harnesses and tool calls
• Familiarity with inspect_ai
• A solid grasp of current AI trends

Benefits

• Fully remote work
• Full-time, flexible hours
• Compensation: $150,000 to $350,000
• Three retreats per year

Company Description

Epoch AI is a research institute that investigates trends in machine learning and the economic consequences of AI. Our mission is to develop a comprehensive, publicly accessible knowledge base on AI that informs policymakers, industry leaders, and society at large.

We strive to achieve both rigor and accessibility to our work, as exemplified by some of our most successful projects, including our database of AI models and our AI trends dashboard. Our body of research includes our work on compute trends (IJCN 2022), data scarcity (ICML 2024), and algorithmic progress (NeurIPS 2024). You can read more about our work and mission on our website and in this Time profile.