[Hiring] Queue Manager / Annotation Lead with Trust & Safety Background (AI Data Ops) @Prolific Academic Ltd

Role Description

We are seeking a Queue Manager / Annotation Lead to join a high-priority initiative in partnership with a leading AI company. You will own day-to-day execution for one or more annotation queues, keeping work flowing, quality on target, and stakeholders unblocked - balancing throughput, SLAs, and accuracy in a live human-data pipeline.

In short, the role involves:

• Owning day-to-day execution of one or more annotation queues
• Managing intake, prioritization, and SLAs
• Running QA and calibration to keep quality on target
• Training and supporting annotators
• Tracking key metrics like throughput and rework rates

For this particular opening, we're specifically looking for candidates with a trust & safety / integrity background - someone who can tell when responses are shallow, low-effort, or phoned-in, and who can keep queues healthy on quality, not just catch bad actors. Think signal-quality analyst rather than a general queue lead.

Qualifications

• 3+ years in high-throughput ops (data labeling, trust & safety, BPO, content ops, CX ops, or similar)
• Trust & safety / integrity background required
• Able to identify shallow, low-effort, or phoned-in responses
• Proven queue management under ambiguity (prioritization + execution)
• Strong quality instincts; can spot systematic errors and close the loop with training/QA
• Comfortable with metrics and tooling (spreadsheets required; SQL/BI a plus)

Requirements

• Manage work queue: intake, triage, prioritization, task assignment, and SLA delivery
• Monitor task queue and backlog control
• Identify and troubleshoot bottlenecks (unclear policy, tooling issues, supply gaps)
• Perform QA: make corrections, provide annotator feedback, flag systematic confusion to Client Delivery Lead
• Monitor quality at project and individual annotator levels
• Drive guideline adherence, run calibration sessions, reduce rework
• Manage annotator performance: unassign underperformers, identify high-performers
• Train new annotators on project guidelines
• Conduct live or recorded webinars as participants join a project
• Host "office hours” for participants
• Oversee dedicated Slack channels: answer questions, provide support, escalate complex issues to Client Delivery Lead
• Track key metrics: throughput, disagreement rates, defect/rework rates
• Provide feedback for process improvements

Benefits

• Competitive pay rates
• Flexible hours
• Ability to work from home