[Hiring] Software Development Engineer in Test @Sofico

Role Description

This role is responsible for making our autonomous AI trustworthy. As an SDET on the AI team, you own quality for non-deterministic, multi-agent systems, which raises questions most testing roles never face:

• How do you prove an AI reached the right answer for the right reason?
• How do you catch a hallucination before a customer sees it?
• How do you guarantee one tenant can never see another's data?
• How do you ensure an agent never exceeds its guard rails?
• How do you prevent quality from quietly drifting when a model or prompt changes?

You will help define how SOFICO Intelligence evaluates agentic AI, designing and building the evaluation harness that underpins our AI governance and our confidence to ship. This is real engineering against a production AI system at scale, not script-running against a static application. You write code to test code, and code to assess the quality of the AI itself.

Qualifications

• Proven experience spanning both software development and quality assurance / quality engineering.
• Strong coding skills, able to program confidently in at least one language (we work primarily in Go and Python).
• Hands-on experience with open-source automated testing tools and frameworks.
• A solid understanding of RESTful APIs and the ability to perform HTTP requests from the command line or a tool like Postman.
• Comfortable working from the command line and reasoning about distributed, concurrent systems.
• Hands-on experience testing, evaluating, or building with LLM-based or agentic systems.
• Demonstrated ability to evaluate non-deterministic AI output against a measurable quality bar.
• Working knowledge of LLM concepts that drive quality.
• A sense of pride and a strong desire to ensure trustworthy, bug-free software ships every day.
• Fluent in English, as the role involves interaction with international teams.

Requirements

• Depth in AI evaluation methodology, such as designing golden datasets, LLM-as-judge scoring, supervisor/critic patterns, or automated regression detection for agentic pipelines.
• Experience with Go, data warehouses / SQL analytics (e.g. Databricks), event-driven systems, or real-time streaming.
• Familiarity with enterprise back-office or ERP environments, including Java back-ends and relational databases (Oracle, MSSQL, Postgres).
• Exposure to the automotive finance, leasing, fleet, or mobility domain.

Benefits

• Competitive benefits, including health care (medical, dental, and vision) for US employees.
• Flexible PTO policy.
• Support for growth through conferences, advanced training, and certifications.