[Hiring] Agent Evals Specialist @Coconut

Role Description

At Coconut 🥥, we connect founders and businesses with top-tier remote talent while prioritizing meaningful, reliable partnerships. We believe success happens when everyone wins. Clients achieve their goals, and our virtual professionals find purpose and growth in their work. We're currently looking for top talent to fill the role below:

• Role: Agent Evals Specialist
• Monthly Rate: $900-1450
• Availability: 40 hours/week
• Time Zone: Preferred overlap anytime during 10AM to 10 PM PST

To streamline your application, please carefully review and complete ALL THREE items below before proceeding to the application form.

What to Prepare:

• English Proficiency Rating – You will need to enter your CEFR English proficiency level in the application form. If you’re unsure of your rating, you can take a quick assessment here: EF SET Quick Check.
• Coconut VA Profile (Resume) – Before submitting your application, please create your Coconut VA profile using this template: Coconut VA Profile Template. ⚠️⚠️⚠️ YOUR RESUME MUST FOLLOW THIS FORMAT AND SHOULD BE IN ENGLISH. This is a test of how keen you are with details and how well you follow instructions. ⚠️⚠️⚠️
• 1-Minute Introduction Video – Record a short introduction video (preferably using Loom) and provide the link in your application. Before you do, check out these video tips and sample videos. Important: By submitting your video, you grant Coconut permission to share it on our platforms to help increase your chances of being matched with a client. Videos with broken or dummy links will result in disqualification.

Qualifications

• Prior work as an AI trainer, tutor, or evaluator (Outlier, DataAnnotation, xAI, Surge, Mercor, Invisible, Toloka, etc.)
• Technical writing, editing, QA, translation, paralegal, or research-assistant background
• Markdown familiarity

Requirements

• Strong written English. Can read dense technical content for hours without losing focus
• Consistent — your scoring on Monday matches your scoring on Friday
• Clear, specific feedback: "section 4 dropped the key requirement from page 17", not "this is confusing"

Industry/Tools Experience

• Slack and internal platform

Benefits

• Competitive Salary – We recognize your skills and experience with a great compensation package.
• 13th Month Pay – Enjoy an extra month’s salary—because who doesn’t love a little bonus?
• Paid Time Off – Recharge with 12 days of PTO to use however you like—vacation, self-care, or just a break from it all!
• US Holidays Off – Enjoy 12 paid US holidays throughout the year.
• Maternity & Paternity Leave – We support growing families with dedicated leave for new parents.
• Comprehensive Healthcare – We’ve got you and your family covered with quality healthcare benefits.
• Life Insurance* – Our full-time HMO plan includes life insurance for extra peace of mind.
• Mental Health Support – Prioritize your well-being with access to mental health consultations and wellness resources.
• Milestone Gifts & Birthday Treats – We celebrate your achievements and make your special day even better!
• Exciting Team Experiences – Think island getaways, virtual town halls, and regional meetups—because work should be fun, too!
• Stay Connected – Be part of a supportive network where you can learn, grow, and connect with amazing people.

Some benefits may vary. Full details will be provided during the onboarding process.