[Hiring] Applied AI Engineer @Dscout

Role Description

At Dscout, we’re building the most flexible and powerful UX research platform on the market—trusted by the world’s top brands in finance, healthcare, consumer goods, and tech. We are expanding our smart and driven team and would love for you to join us.

We're looking for an Applied AI Engineer with 2-5 years of experience building and shipping AI systems used by professionals at enterprise. You're comfortable working with modern LLM-based systems and agentic workflows, and you know how to turn powerful models into reliable product features.

What you'll do:

• Own the production improvement loop across agent behavior, customer and operator feedback, evaluation, experimentation, and verified business outcomes.
• Instrument agent workflows so model interactions, tool use, decisions, failures, human edits, and downstream outcomes can be understood in context.
• Define meaningful quality standards, representative evaluation datasets, regression coverage, and production monitoring.
• Investigate why agents underperform across context, knowledge, instructions, tools, routing, guardrails, or workflow design.
• Design and ship targeted behavior improvements, including changes to prompting, context construction, decision logic, tool use, and human-review paths.
• Build backend services, APIs, data models, and feedback pipelines that make agent behavior observable, steerable, and reproducible.
• Run controlled experiments, production replays, or staged rollouts to measure whether changes improve quality and downstream business results.
• Partner with Product, Data Science, and Sales to prioritize high-value problems and define customer and business success.
• Ship with appropriate safeguards for privacy, security, reliability, human oversight, and safe operational rollout.

Qualifications

• 2-5 years of software engineering experience, with hands-on experience building or operating LLM-powered features or agents in production.
• Fluency with prompting and context engineering as an engineering discipline.
• Experience building or maintaining evaluation harnesses for AI systems.
• Genuine comfort with non-determinism.
• Experience running experiments (A/B, staged rollouts, production replay) to validate whether a change actually improved outcomes.
• A track record of shipping features real users depended on, and owning what happened after launch.
• A high-agency mindset.
• Comfort using AI coding tools (Cursor, Claude Code, Copilot, or similar).

Requirements

• Experience with voice or real-time conversational AI systems (nice to have).
• Familiarity with LLM observability/tracing tools (e.g., Braintrust, LangSmith, Datadog LLM Observability) (nice to have).
• Experience with agentic orchestration frameworks (LangChain/LangGraph or similar) (nice to have).
• Exposure to MCP-based tooling or agentic data workflows (nice to have).

Benefits

• A strong and competitive compensation package with a built-in bonus and equity program.
• An incredible and progressive benefits package (for both you and your dependents) to support work/life balance, including flexible PTO, 15 company holidays, 12 weeks of paid parental leave, 401k match, and much more.
• An education stipend to support your growth & development, and a remote work stipend.
• A company that is open and transparent with our team.

Company Description

Dscout is a team of passionate, empathetic, and curious professionals. As a recognized leader in the Forrester Wave, we’re at the cutting edge of experience research technology. We prioritize learning, sharing, and building, and we deeply value being a diverse and inclusive team.