[Hiring] Personal Account Generalist Rater @Turing

Role Description

We are seeking Personal account generalist raters to participate in Gemini Personalization Evals. In this role, raters will perform generalist evaluation work using personalized prompts designed to retrieve context and data across connected Google apps.

• Evaluating AI-generated responses based on your professional email history and business account activity.
• Assessing whether responses are relevant, accurate, helpful, and appropriately personalized.
• Identifying issues related to incorrect personalization, unsupported assumptions, or irrelevant recommendations.
• Comparing multiple AI responses and determining which provides the better overall user experience.
• Providing clear, detailed, and well-reasoned evaluation feedback to support model improvements.
• Following project guidelines to ensure consistent, accurate, and high-quality evaluations.
• Maintaining data privacy and adhering to all project confidentiality requirements throughout the evaluation process.

Qualifications

• Must be currently based in Canada.
• Must be willing and able to connect their Google apps (e.g., Gmail, Google Photos, Google Calendar, Google Drive, etc.) to Gemini to support data retrieval and personalized prompt evaluations.
• Bachelor's degree or equivalent practical experience in any field.
• Experience in AI evaluation, data annotation, content review, quality assurance, or a related analytical role is preferred but not required.

Requirements

• Perform generalist rating and evaluation tasks, assessing model performance, context retrieval, and response quality across connected apps.
• Active use of standard Google workspace/consumer apps with sufficient personal data/history to generate and evaluate retrieval prompts.
• Ability to evaluate nuanced AI responses and identify strengths and weaknesses in personalization quality.
• Strong ability to review AI-generated responses and identify subtle issues or inconsistencies.
• Excellent written communication skills for providing clear and structured evaluation feedback.
• Self-motivated and able to work independently in a remote environment.
• Desktop or laptop with a reliable internet connection.

Benefits

• Commitments Required: at least 4 hours per day and up to 40 hours per week with 4 hours of overlap with PST.
• Engagement type: Contractor
• Engagement Length: up to 16 weeks

Evaluation Process

• Shortlisted candidates will be sent a Job Interest Form.
• Final selected candidates will be contacted with the next steps, including consent and onboarding requirements.

Consent & Compliance

• Must be willing to grant the necessary app integration permissions required for the project and complete all standard consent forms and non-disclosure agreements.