[Hiring] AI Agent Evaluator @YO AI Labs

Role Description

We are seeking detail-oriented professionals to evaluate AI agents performing everyday tasks. You'll test two AI agents side by side, assess their performance, and provide feedback to improve their reliability and user experience.

• Test AI agents on everyday tasks, including shopping, travel, bookings, bill payments, email follow-ups, and forms.
• Approve or stop agents at key steps.
• Screen-record each testing session.
• Score task completion, quality, control issues, user satisfaction, and time.
• Provide clear written feedback on agent performance and usability.

Qualifications

• Based in the United States with a bachelor's degree.
• Regularly use AI tools in daily life.
• 2–3 years of experience, preferably in engineering, business management, or office operations.
• Strong attention to detail and ability to follow instructions.
• Excellent written communication and documentation skills.
• An active personal AI account (Free, Go, or Plus; not Pro) and a smartphone capable of screen recording.

Requirements

• Relevant domain expertise. No coding or AI development experience is required.

Selection Process

• Submit an application and complete screening questions.
• Attend a 15–30-minute interview.
• Complete profile review and onboarding.

Project Details

• Task duration: Approximately 1.5 hours per run, including evidence and scoring.
• Kickoff: Monday, October 12.
• Onboarding: Before the project begins, subject to scheduling.