[Hiring] QA Analyst @CodeRoad

Role Description

We’re looking for a QA Analyst specialized in modern AI systems. You’ll be responsible for validating not only APIs and performance, but also the behavior and reliability of AI-driven outputs. This role goes beyond traditional QA — you’ll test intelligent systems, not just deterministic ones.

Key Responsibilities

• Design and execute test strategies for AI-powered APIs and workflows
• Validate agent inputs/outputs and ensure consistency and reliability
• Perform API testing, exploratory testing, and automated testing
• Run performance, load, and bulk testing scenarios
• Collaborate with engineers to identify edge cases in AI behavior
• Contribute to testing frameworks for LLM-based systems

Qualifications

• 3+ years of experience in QA (Automation + Manual)
• Strong API Testing: Expert level with Postman, Insomnia, or RestAssured
• Automation Specialist: Proficient in Playwright or Selenium for end-to-end flows
• Performance & Scale: Experience with Load/Bulk testing using Locust or JMeter
• AI/LLM Validation: Familiarity with testing Agentic workflows, validating AI payloads, and using tools like LangSmith
• Observability: Exposure to OpenTelemetry for tracing and system monitoring
• Analytical Rigor: Ability to design test cases for non-linear, agentic AI tasks
• Advanced English: Fluent for documentation and global collaboration

Nice to Have

• Experience testing AI/ML systems
• Familiarity with LangSmith or similar tools
• Experience with MLFlow or model evaluation frameworks

Benefits

• 100% Remote
• Holidays off
• Paid Time Off
• Health insurance assistance program
• Competitive USD compensation
• Strong team culture and collaborative environment
• Ongoing training and growth opportunities