[Hiring] Healthcare Subject Matter Expert @This Opportunity

Role Description

We are seeking experienced Healthcare Subject Matter Experts (SMEs) to help evaluate and improve advanced AI systems through expert-level question development, clinical content evaluation, and detailed rubric design. In this remote contractor role, you will:

• Create challenging healthcare questions that require genuine professional expertise.
• Evaluate AI-generated clinical content.
• Develop rigorous scoring rubrics that define what an accurate, complete, safe, and professionally appropriate response should contain.
• Identify where advanced AI models produce incomplete, inaccurate, poorly reasoned, overconfident, or clinically inadequate responses.

Key Responsibilities

• Create expert healthcare questions:
  • Develop challenging, specialized questions designed to test advanced AI systems and expose limitations in clinical reasoning.
• Design detailed evaluation rubrics:
  • Define the facts, reasoning steps, clinical considerations, caveats, and standards required for a high-quality response.
• Identify critical criteria:
  • Separate essential clinical reasoning and safety considerations from supplementary information.
• Evaluate AI-generated responses:
  • Assess outputs for accuracy, completeness, reasoning quality, safety, professional appropriateness, and clinical relevance.
• Identify subtle failures:
  • Detect unsupported assumptions, missing considerations, inappropriate certainty, clinically significant omissions, and reasoning gaps.
• Critique peer-created rubrics:
  • Review rubrics developed by other healthcare professionals and identify ambiguity, redundancy, incorrect expectations, missing criteria, or weak scoring structures.
• Develop reference answers:
  • Create expert examples, explanations, critiques, and gold-standard response content.
• Support AI training workflows:
  • Produce questions, rubrics, evaluations, critiques, and reference-answer pairs supporting RL and SFT workflows.
• Maintain clinical quality:
  • Ensure evaluation content uses appropriate healthcare terminology, professional reasoning, and evidence-aware judgment.

Qualifications

• Professional degree, license, credential, or recognized qualification in medicine, dentistry, pharmacy, nursing, or another healthcare discipline.
• 3+ years of professional or clinical experience.
• Demonstrated expertise in a clearly defined healthcare specialty or subspecialty.
• Strong professional proficiency in English (C1 minimum).
• Ability to develop challenging clinical or professional questions requiring genuine expertise and nuanced judgment.
• Strong written communication and the ability to develop detailed evaluation criteria.
• Ability to distinguish minor omissions from errors that could materially affect clinical quality or safety.
• Ability to evaluate AI-generated content for accuracy, completeness, reasoning quality, safety, calibration, and clinical relevance.
• Ability to independently produce consistent, high-quality written work.

Preferred Qualifications

• Senior-level or specialist healthcare experience.
• Experience with medical education, examination writing, competency assessment, clinical guidelines, teaching, peer review, or quality assurance.
• Experience with AI evaluation, RLHF, SFT, data annotation, model benchmarking, or rubric-based scoring.
• Strong understanding of clinical reasoning and evidence-based professional judgment.

Areas of Expertise

• Medicine
• Dentistry
• Pharmacy
• Clinical Pharmacology
• Nursing
• Surgery
• Radiology
• Pathology
• Emergency Medicine
• Clinical Reasoning
• Medical Education
• Healthcare Quality & Assurance
• AI Evaluation
• Rubric Design
• Expert Review

What You'll Work On

Your work may involve evaluating AI responses across complex healthcare scenarios and determining whether an answer demonstrates the level of accuracy, completeness, reasoning, and professional judgment expected from a qualified healthcare expert. You will help define what separates a superficial response from an expert-level healthcare answer by identifying required concepts, appropriate reasoning pathways, important caveats, acceptable alternatives, and critical errors.

Why Join This Opportunity?

• Shape the Future of AI: Apply your healthcare expertise to help improve how advanced AI systems reason about and communicate healthcare information.
• Work Remotely: Work remotely with flexibility around your schedule and professional commitments.
• Use Your Professional Expertise: Your clinical or healthcare knowledge directly contributes to the development and evaluation of next-generation AI systems.
• Weekly Payments: Receive payment for approved work on a weekly basis.
• Community Rewards: Referral opportunities may provide additional bonus income.

Position Details

• Role: Healthcare Subject Matter Expert – Rubric Design
• Work Arrangement: Remote
• Location: United States
• Employment Type: Contract
• Compensation: Up to $100/hour
• Annualized Equivalent: Up to $192,000/year
• Interview: Required
• Language: English
• Experience: 3+ years professional or clinical experience

Skills

• Clinical reasoning
• Rubric design
• AI evaluation
• Medical subject matter expertise
• Technical writing
• English proficiency (C1)
• Critical analysis
• Healthcare guidelines knowledge