[Hiring] Senior Applied Scientist @Caseware

Role Description

We are building the agentic platform that powers Caseware's next generation of audit and accounting products, and applied science is how we measure and improve quality across our agents. This is a role for someone who thinks in experiments, has built evaluations for LLM-based systems, and wants to build the tools that other teams and our customers rely on to ship their own agents.

• Design and run experiments that turn ambiguous quality questions into measurable results.
• Own the evaluation methodology that internal teams and customers depend on.
• Build AI-based capabilities behind our end-to-end agent builder.
• Work on self-learning, self-improving systems that get better both offline and online.
• Influence technical direction and mentor others at the staff level.

Qualifications

• 6+ years (senior) to 8+ years (staff) of professional experience in applied science, machine learning, research, or data-intensive engineering.
• 2+ years working on production GenAI or LLM-based systems.
• Proven experience designing and building evaluation frameworks for ML or LLM systems.
• A strong experimentation mindset.
• Ability to build and operate your own production tooling and services, ideally on AWS.
• Strong understanding of GenAI system tradeoffs.
• Strong foundation in probability and statistics.
• Strong English language communication and collaboration skills.
• Comfortable operating in fast-moving environments with ambiguity and evolving requirements.

Requirements

• Design and run experiments that measure and improve the quality of LLM-based applications and agents.
• Build out and own the evaluation methodology in practice.
• Build AI-based tools and capabilities for the end-to-end agent builder developer experience.
• Build synthetic data generation and eval-builder capabilities.
• Contribute to applied science on agentic memory.
• Help ensure memory promotion and demotion criteria uphold legal and contractual obligations.
• Design experiments and evaluations that demonstrate agent quality improves with customer use.
• Translate advances in GenAI into practical, maintainable capabilities.
• Influence technical direction through RFCs and design reviews.
• Mentor other scientists and engineers.

Benefits

• Indefinite contract with all legal benefits.
• Prepaid medicine.
• Life insurance and funeral assistance.
• Internet allowance.
• Home office stipend.
• Competitive compensation — above the market average.
• 100% remote work environment and an excellent work-life balance.
• 5 Personal Time Off days per year.
• Sick Leave Top up to total 100% of salary paid by the employer from Day 3 to 90.
• Recognition Award, additional paid time off in recognition of the corresponding year of service.
• Upgrade vacation starting at 5 years of service.
• Opportunity to work for a growing global SaaS leader company.
• A culture that promotes independence, innovation, trust, and accountability.
• Open space to be creative, innovative and strategize for the future.
• Mentorship by highly experienced professionals.
• Budget for training.
• AI-first environment.