AI Evaluation Expert - Rubric Specialist - AI Trainer @Mercor
Artificial Intelligence
Salary $40–$75/hour
Remote Location
πŸ‡ΊπŸ‡Έ USA Only
Employment Type part-time
Posted 3d ago

[Hiring] AI Evaluation Expert - Rubric Specialist - AI Trainer @Mercor

3d ago - Mercor is hiring a remote AI Evaluation Expert - Rubric Specialist - AI Trainer. πŸ’Έ Salary: $40–$75/hour πŸ“Location: USA

Role Description

As an AI Evaluation Rubric Reviewer, you will:

  • Review AI-drafted tasks and evaluate model attempts for fairness and accuracy.
  • Assess if the grading rubric aligns with the prompt and correct any unfair criteria.
  • Flag tasks that cannot be saved and provide concise feedback on each evaluation.
  • Analyze long agent transcripts to identify errors and improve AI model performance.
  • Ensure rubric structures are correct per guidelines and enhance training data quality.

Qualifications

  • Substantial experience writing or reviewing rubrics for LLM evaluations.
  • Ability to read long agent transcripts and identify errors.
  • Precise writing skills and a commitment to data verification.

Company Description

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include:

  • Benchmark
  • General Catalyst
  • Peter Thiel
  • Adam D'Angelo
  • Larry Summers
  • Jack Dorsey
Before You Apply
️
πŸ‡ΊπŸ‡Έ Be aware of the location restriction for this remote position: USA Only
β€Ό Beware of scams! When applying for jobs, you should NEVER have to pay anything. Learn more.
AI Evaluation Expert - Rubric Specialist - AI Trainer @Mercor
Artificial Intelligence
Salary $40–$75/hour
Remote Location
πŸ‡ΊπŸ‡Έ USA Only
Employment Type part-time
Posted 3d ago
Apply for this position
Did not apply βœ“
Applied βœ“
Sent Follow-Up βœ“
Interview Scheduled βœ“
Interview Completed βœ“
Offer Accepted βœ“
Offer Declined βœ“
Application Denied βœ“
Unlock 140,000+ Remote Jobs
️
πŸ‡ΊπŸ‡Έ Be aware of the location restriction for this remote position: USA Only
β€Ό Beware of scams! When applying for jobs, you should NEVER have to pay anything. Learn more.
Apply for this position
Did not apply βœ“
Applied βœ“
Sent Follow-Up βœ“
Interview Scheduled βœ“
Interview Completed βœ“
Offer Accepted βœ“
Offer Declined βœ“
Application Denied βœ“
Unlock 140,000+ Remote Jobs
Γ—
Apply to the best remote jobs
before everyone else

Access 140,000+ vetted remote jobs and get daily alerts.

4.9 β˜…β˜…β˜…β˜…β˜… from 500+ reviews

⚑ 142,884+ remote jobs, refreshed hourly

πŸ”” Real-time alerts: Apply first, direct to employer

πŸ›‘οΈ Vetted companies, no scams, true remote only

Unlock All Jobs Now

Maybe later