[Hiring] Research Evaluation Specialist @RemoteJobsOne

Role Description

This is a fully remote position, open to candidates based in Canada.

We are engaging Research Evaluation Specialists (PhD / Researcher / Professor) to contribute expert insight in developing rigorous test material for advanced AI model evaluation as part of a customer's project. In this role, you'll apply your expertise to help train next-generation AI systems. Your work will shape how models learn, reason, and perform through high-quality, real-world input. No prior experience in AI is required — your domain knowledge is what matters.

Scope of Work

• Create original, high-difficulty question-and-answer pairs at the cutting edge of your discipline to rigorously assess AI model capabilities.
• Source, verify, and thoroughly document answers using primary sources and authoritative references, including clear citations and reasoning.
• Design questions demanding advanced reasoning, methodological nuance, or synthesis across multiple sources, ensuring resistance to oversimplified or shortcut solutions.
• Test questions against AI systems, identify items that are not sufficiently challenging, and iteratively increase complexity while maintaining accuracy.
• Write each question and answer with high precision and clarity, eliminating ambiguity and supporting defensible responses.
• Integrate reviewer feedback and consistently uphold all project guidelines and quality standards throughout your deliverables.

Qualifications

• PhD completed, active candidacy, or equivalent research experience as a specialist, researcher, or professor in your domain.
• Demonstrated track record of scholarly research, in-depth subject matter expertise, and familiarity with primary literature.
• Exceptional analytical thinking, close attention to detail, and strong written communication skills in English.
• Experience sourcing and triangulating information from authoritative references.
• Proven self-direction and reliability in delivering expert-level work independently within remote engagements.
• Skill in composing challenging, original, and methodologically sound questions.
• Experience in AI training or evaluation is advantageous but not required.

Requirements

• Compensation is output-based; experts are paid per task that meets the project specifications.
• The time required to complete work may vary depending on the expert’s experience and workflow.
• Minimum submission requirements apply. Experts must submit a minimum of tasks per week.

Benefits

• We typically fill roles within 48 hours and are looking for experts ready to jump in right away.
• If selected, we expect you to start your first tasks within 24–48 hours of completing onboarding.

How to apply

Apply here with your Indeed profile. We will then email you a link to complete your application on the hiring platform — your application is only considered once that final step is done.