[Hiring] AI Test Engineer @INSPYR Solutions

Role Description

We are seeking a highly motivated AI Test Engineer to help establish and scale our AI testing and evaluation capability. This role will work closely with AI engineers, platform teams, and product stakeholders to build automated testing frameworks, evaluate AI-driven solutions, and ensure the quality, reliability, and performance of AI-enabled products.

The AI Test Engineer will be hands-on contributors responsible for:

• Creating automated tests
• Executing evaluations against AI systems
• Analyzing model outputs
• Defining best practices for AI quality assurance

This role requires a strong combination of software testing expertise, automation engineering, data analysis, and an understanding of modern AI and generative AI technologies.

Qualifications

• Bachelor's degree in Computer Science, Information Systems, Software Engineering, Data Science, or a related technical field.
• 3+ years of experience in software testing, quality engineering, test automation, or AI/ML testing.
• Experience designing and executing automated test strategies for complex software applications.
• Strong programming and scripting skills in Python.
• Experience building automated test frameworks and test pipelines.
• Experience working with APIs, automated integration testing, and CI/CD environments.
• Understanding of machine learning, generative AI, large language models (LLMs), or AI-powered applications.
• Experience performing data validation, test result analysis, and defect investigation.
• Familiarity with modern testing methodologies, quality engineering practices, and software delivery lifecycles.
• Knowledge of statistical analysis and evaluation techniques used to validate system performance.
• Experience using source control systems and collaborative development practices.
• Strong analytical, problem-solving, and troubleshooting skills.
• Excellent written and verbal communication skills.
• Ability to work independently in a fast-paced, agile environment while collaborating effectively across teams.

Requirements

• Help design, develop, and maintain AI testing and evaluation frameworks for generative AI, predictive AI, and AI-powered developer experience (DevEx) solutions.
• Create and execute automated test suites that validate AI model behavior, output quality, accuracy, reliability, and performance.
• Develop automated evaluation pipelines that integrate with CI/CD workflows and support continuous AI quality validation.
• Perform functional, regression, performance, and scenario-based testing of AI-enabled applications and services.
• Analyze AI-generated outputs to identify defects, inconsistencies, hallucinations, bias, performance issues, and other quality concerns.
• Build and maintain test datasets, benchmark scenarios, prompt libraries, and evaluation artifacts used to assess AI systems.
• Collaborate with software engineers, AI engineers, and platform teams to improve model quality and testing coverage.
• Define, measure, and report AI quality metrics such as accuracy, relevance, latency, consistency, robustness, and user satisfaction.
• Support root-cause analysis, troubleshooting, and validation activities for AI-related defects and incidents.
• Document test strategies, test results, evaluation methodologies, and recommendations for technical and non-technical stakeholders.
• Contribute to AI governance and responsible AI efforts by supporting evaluation activities related to reliability, fairness, transparency, and safety.
• Support delivery of strategic DevEx initiatives and AI-enabled platform capabilities across the software development lifecycle.

Company Description

INSPYR Solutions provides Equal Employment Opportunities (EEO) to all employees and applicants for employment without regard to race, color, religion, sex, national origin, age, disability, genetics, or any other protected status. INSPYR Solutions complies with all applicable laws governing nondiscrimination in employment in every location in which the company has facilities.

Applicants requiring reasonable accommodation during the application or interview process should contact [email protected] for assistance.

Information collected and processed through your application with INSPYR Solutions (including any job applications you choose to submit) is subject to INSPYR Solutions' Privacy Policy and INSPYR Solutions' AI and Automated Employment Decision Tool Policy: https://www.inspyrsolutions.com/policies/ .

By submitting an application, you are consenting to being contacted by INSPYR Solutions through phone, email, or text.