[Hiring] Expert Rater @Welocalize

Role Description

This role will play a crucial role in analyzing and improving AI-driven online advertising platforms.

• Rate Client-provided ads following pre-defined guidelines
• Evaluate advertisements based on specific quality, relevance, and compliance criteria provided by the client
• Ensure ratings are objective, consistent, and aligned with the predefined rating standards
• Identify any guideline ambiguities and apply critical thinking when assessing borderline cases
• Maintain a high level of accuracy and efficiency, meeting daily/weekly rating quotas
• Review and refine rating guidelines, identifying areas for improvement and recommending updates to enhance accuracy, consistency, and overall rating quality
• Curate high-quality datasets ("golden sets") by selecting and validating top-tier rated ads, which can serve as benchmarks for AI model training and human rater calibration
• Collaborate with cross-functional teams (e.g., quality assurance, data analysts, and AI training teams) to ensure rating processes align with evolving industry and client needs
• Analyse trends and patterns in ad ratings, providing insights that contribute to guideline optimizations and overall rating effectiveness
• Assist in training and mentoring new raters, ensuring they understand guidelines and best practices for accurate evaluations

Qualifications

• Experience in AI model relevance, such as data labeling, linguistic annotation, relevance rating, etc.
• Analytical and data-driven with a proven track record of data-based decision making
• Strong interest and understanding of digital marketing and advertising, especially as it relates to artificial intelligence
• Ability to effectively summarize complex information, questions, and solutions for various audiences
• Experience researching and gathering insights, and understanding complex problems across a variety of topics
• Ability to define, explain, and defend points of view in group discussions
• Comfortable reviewing and discussing a variety of topics with a willingness to research and understand to support consensus decisions
• Comfortable with ambiguity in guidelines, workflows, and processes, and motivated to help establish those systems

Requirements

• Native level fluency in the target language
• Excellent understanding of both written & spoken English
• A 4 year university degree and/or at least 2 years work experience in one or more of the following: AI-based relevance rating, marketing/advertising, linguistics, content creation, writing, tech-related quality assurance, data labeling, or linguistic annotation

Company Description