[Hiring] Business Domain Expert — AI Model Evaluation @the Opportunity

Role Description

We are seeking experienced professionals in Business Operations, Marketing, Human Resources, and Accounting to participate in a structured AI model evaluation pilot. You will complete one realistic professional task within your area of expertise and evaluate five AI-generated attempts at the same task. Your professional judgment will help assess how effectively advanced AI models perform real-world professional work. This is a one-time evaluation study, rather than an ongoing production role.

• Sign a confidentiality agreement before receiving study materials.
• Complete one realistic, domain-specific task using the provided source materials.
• Review five AI-generated outputs for the same task.
• Rank the outputs from 1–5 with no ties.
• Score each output from 0–100 based on its quality and effectiveness.
• Write clear, evidence-based rationales referencing specific parts of each output.
• Provide structured feedback on the task and study design.

Qualifications

• 5+ years of hands-on professional experience in at least one of the four target domains.
• Current or recent professional experience in your area of expertise.
• Strong professional judgment and ability to assess the quality of real-world business work.
• Excellent written communication skills.
• Ability to provide specific, clear, and evidence-based reasoning.
• Reliable availability to complete the study within a short turnaround window.

Requirements

• No rubric or predefined answer will be provided.
• Your professional judgment is the basis of the evaluation.
• The use of LLMs, AI assistants, or other AI tools is prohibited throughout the study.
• Evaluations must be based on your own professional knowledge and experience.

Benefits

• Independent contractor engagement.
• Fully remote with flexible scheduling.
• One-time, short-term evaluation project.
• Payments are made weekly based on services rendered.