[Hiring] AI Safety Evaluator @Welo Global

Role Description

We are seeking experienced bilingual evaluators to support a multilingual AI safety project focused on evaluating model responses across culturally specific prompt-image datasets. This project involves applying a structured safety rubric to assess AI-generated responses for appropriateness, safety, and reliability within the target locale’s cultural context. Each language stream will process approximately 1,000 prompt-image pairs. Every item will receive two independent evaluations, with arbitration applied in cases of disagreement. Evaluations will primarily be documented in English, with a defined in-language sample.

Project Details

• Location: Remote – South Korea
• Team: Welo Data – AI Services
• Engagement Type: Freelance – Remote
• Start Date: As soon as possible
• Duration: 2-3 weeks
• Weekly Commitment: 20–40 hours per week
• Schedule Options:
  • 4 hours per day, Monday–Friday
  • 2 hours per day, Monday–Friday + 10 weekend hours
• Hourly rate: 22 USD

Responsibilities

• Evaluate AI-generated responses using a structured safety rubric
• Complete two independent evaluations per item
• Provide concise, well-structured rationales in English
• Participate in calibration sessions
• Support arbitration when evaluation discrepancies occur
• Maintain quality and throughput targets during the evaluation window

Qualifications

• Fluency in the target language and English
• Deep cultural understanding of the target locale
• Strong written English skills for documentation and rationales
• Prior experience in safety evaluation, policy review, content moderation, or rubric-based assessment preferred
• Ability to apply detailed guidelines consistently
• Strong analytical skills and attention to nuance
• Reliable availability during the production window
• Priority may be given to contributors who previously worked on prompt-image or similar evaluation projects to reduce onboarding time and maintain continuity

Disclaimer

This role involves working with explicit and sensitive content. Applicants should be comfortable working with adult material in a professional capacity. Please apply only if you fully understand and are prepared for the nature of this role.

Benefits

• ✨ Limitless Flexibility: Project-based opportunities that fit your availability. Choose when and how much you want to contribute—fully remote, with complete autonomy.
• 🌱 Limitless Growth: Optional access to AI and Large Language Model workshops designed specifically for professionals like you. No coding required—just your expertise.
• 🌍 Limitless Support: Be part of a global contributor community with responsive guidance and support.
• 💡 Real Impact: Apply your expertise in the Legal field to influence the AI systems shaping the future of your industry—while collaborating with data professionals and expanding your skills.

How to Apply?

Apply now by answering a few quick questions to join our database and become part of our growing community.