[Hiring] Medical Expert AI Evaluation Specialist @Weekday AI

Role Description

Join a leading AI research initiative focused on advancing healthcare-focused artificial intelligence. We are seeking experienced medical professionals to contribute their clinical expertise toward training and evaluating next-generation AI models capable of sophisticated medical reasoning.

In this role, you will help:

• Create realistic clinical scenarios
• Evaluate AI-generated responses using evidence-based medical standards
• Provide expert insights that improve how AI systems reason about diagnosis, treatment, and patient care

This opportunity is ideal for physicians across a broad range of medical specialties who are passionate about applying their expertise to the future of healthcare technology.

Key Responsibilities

• Develop Clinical Scenarios
  • Create realistic, specialty-specific clinical cases that reflect real-world medical practice
  • Design prompts covering areas such as:
    • Diagnostic reasoning
    • Treatment planning
    • Risk stratification
    • Differential diagnosis
    • Guideline-based clinical decision-making
• Create Reference-Quality Solutions
  • Produce high-quality clinical responses that reflect attending physician-level reasoning
  • Ensure recommendations align with current evidence-based practices and accepted clinical guidelines
• Evaluate AI Performance
  • Assess AI-generated medical responses using structured evaluation frameworks and scoring rubrics
  • Identify strengths, weaknesses, omissions, and reasoning errors
  • Provide detailed written feedback to improve model performance and clinical accuracy
• Collaborate with Clinical Experts
  • Participate in onboarding sessions and specialty calibration meetings
  • Work with multidisciplinary research teams to maintain consistency and quality across medical evaluations

Qualifications

• Attending Physicians
  • Board-certified physician with a current, active, and unrestricted medical license
• Resident Physicians
  • Currently in the final year of an accredited residency program
  • Recent residency graduates must be board-eligible
• Fellows
  • Board-certified or board-eligible in their primary specialty
  • Current, active, and unrestricted medical license
• Physicians from a wide range of specialties are encouraged to apply, including but not limited to:
  • Family Medicine
  • Pediatrics
  • Anesthesiology
  • General Surgery
  • ENT (Otolaryngology)
  • Physical Medicine & Rehabilitation (PM&R)
  • Occupational Medicine
  • Other medical and surgical specialties

Preferred Qualifications

• Strong understanding of evidence-based medicine and clinical practice guidelines
• Excellent written communication skills
• Experience with medical education, clinical research, peer review, or healthcare quality initiatives
• Interest in artificial intelligence and healthcare innovation

Engagement Details

• Fully remote with 100% asynchronous work
• Typical commitment of approximately 20 hours per week, with opportunities for additional hours based on project needs
• Flexible schedule
• Weekly payments
• Rolling onboarding with ongoing project opportunities
• Engagement duration is reviewed periodically based on project requirements and performance

Why Join

• Help shape the future of AI-powered healthcare
• Apply your clinical expertise to improve advanced medical reasoning systems
• Collaborate with leading AI researchers and healthcare professionals
• Contribute to high-impact projects that advance patient-centered artificial intelligence

Equal Opportunity Statement

All qualified applicants will be considered without regard to legally protected characteristics. Reasonable accommodations are available upon request.

Contract Information

• Independent contractor engagement
• Fully remote work completed on your own schedule
• Project duration may be extended, modified, or concluded based on business needs and individual performance
• Work does not involve access to confidential or proprietary information from any employer, client, or institution
• Weekly payments are processed through supported payment platforms based on approved work completed
• Please note that visa sponsorship is not available for this opportunity