[Hiring] Trust & Safety Analyst @Welo Data

Role Description

We are seeking detail-oriented Trust & Safety Analysts – AI Evaluation to review and evaluate AI-generated outputs using content safety policies, project guidelines, and expert human judgment. In this role, you will:

• Assess complex AI outputs and provide accurate, consistent evaluations that help improve the safety and quality of AI systems.
• Review situations where context, language, intent, and cultural understanding are important in determining the appropriate evaluation.
• Provide high-quality human feedback and evaluation data that supports the training and improvement of AI systems.

Qualifications

• Educational background or equivalent experience in Trust & Safety, Content Safety, Policy, Linguistics, Communications, Journalism, Research, Law, Social Sciences, or a related field.
• Experience in trust and safety, content moderation, content policy, AI evaluation, data annotation, quality assurance, or a related field.
• Strong understanding of content safety concepts and the ability to apply detailed policies and guidelines.
• Experience reviewing or evaluating complex content where context and intent are important.
• Ability to apply consistent judgment while separating personal opinions from defined policies and project requirements.
• Awareness of cultural and linguistic differences and their impact on content interpretation.
• Strong analytical and critical-thinking skills.
• Ability to make informed decisions in complex or ambiguous situations.
• Strong written English comprehension and communication skills.
• Excellent attention to detail and the ability to maintain consistency across high volumes of work.
• Familiarity with AI-generated content, AI evaluation, human feedback, RLHF, or data annotation is preferred.

Requirements

• Review and evaluate AI-generated outputs according to defined content safety policies and project guidelines.
• Apply expert human judgment to assess complex, ambiguous, or context-dependent situations.
• Grade, classify, label, or annotate AI outputs accurately and consistently.
• Consider context, language, intent, and cultural differences when evaluating content.
• Identify potential content safety or policy concerns based on defined guidelines.
• Evaluate difficult cases and make informed decisions when the appropriate outcome depends on multiple factors.
• Provide accurate human feedback to support AI system training and evaluation.
• Identify unclear or unusual cases and flag them according to defined project processes.
• Document decisions and supporting reasoning clearly when required.
• Apply detailed project guidelines consistently across assigned tasks.
• Maintain high levels of quality, accuracy, consistency, and attention to detail.

Benefits

• Contract Type: Freelance, with the potential to convert to a full-time role.
• Pay Rate: US$3 per hour
• Location: India
• Language: English

Company Description

Join our team at Welo Data and embark on a journey of growth and innovation.