[Hiring] QA Automation @TEKsystems

Role Description

We are seeking 3 Evaluation Analysts to assess the performance of AI models tasked with implementing web features. Your work directly informs whether AI-generated code is correct, whether the instructions given to the models are clear, and whether the testing frameworks used to evaluate them are fair and reliable.

Core Responsibilities

• Model Capability – Assess how well the AI performed each task by reviewing generated transcripts and results.
• Bug Discovery Value – Identify patterns in AI code failures to understand why the model is making specific mistakes.
• Score Health – Ensure tests are properly configured to apply correct scores during evaluation.
• Task Specification Quality – Verify that prompts given to the AI are clear, correct, and technically precise (e.g., consistent variable names).
• Test-by-Test Analysis – Evaluate the quality of automated tests using metrics like precision and recall to ensure they accurately measure AI performance.
• Platform Issues – Report bugs or problems within the evaluation system itself, particularly with automated browser testing services.

Qualifications

• Expertise in finding patterns and issues in generative AI/LLM outputs
• Direct experience with labeling and scoring frameworks
• Experience writing Playwright tests
• Strong analytical and problem-solving skills
• Ability to interpret model behavior and articulate failure modes clearly

Preferred Qualifications

• Advanced analytical credentials — highly relevant for interpreting model behavior
• Familiarity with web development concepts (HTML, CSS, JavaScript)
• Experience with automated testing and grading systems
• Background in quality assurance or evaluation methodology

Requirements

• Playwright

Job Type & Location

This is a Contract position based out of San Francisco, CA.

Pay and Benefits

The pay range for this position is $35.00 - $50.00/hr. Eligibility requirements apply to some benefits and may depend on your job classification and length of employment. Benefits are subject to change and may be subject to specific elections, plan, or program terms. If eligible, the benefits available for this temporary role may include the following:

• Medical, dental & vision
• Critical Illness, Accident, and Hospital
• 401(k) Retirement Plan – Pre-tax and Roth post-tax contributions available
• Life Insurance (Voluntary Life & AD&D for the employee and dependents)
• Short and long-term disability
• Health Spending Account (HSA)
• Transportation benefits
• Employee Assistance Program
• Time Off/Leave (PTO, Vacation or Sick Leave)

Workplace Type

This is a fully remote position.

Application Deadline

This position is anticipated to close on May 29, 2026.