[Hiring] STEM & Technical Specialist @Xperteez Technology

Role Description

In this hourly, remote contractor role, you will work as a STEM & Technical Subject Matter Expert to:

• Create challenging expert-level questions.
• Evaluate AI-generated technical content.
• Design detailed rubrics that define what a correct, rigorous, and expert-quality answer should contain.

You will:

• Develop problems within your area of expertise that test advanced reasoning rather than simple factual recall.
• Identify weaknesses in frontier AI models, including:
• Incorrect assumptions
• Incomplete reasoning
• Mathematical or scientific errors
• Missed edge cases
• Answers that appear plausible but fail under expert scrutiny
• Translate complex technical judgment into clear, specific, and measurable evaluation criteria.
• Critique and improve rubrics developed by other experts.

This role is a fast-growing AI Data Services company and subsidiary of SuperAnnotate, delivering training data for many of the world’s largest AI companies and foundation-model labs.

Your technical expertise directly helps improve the world’s premier AI models by making their reasoning more accurate, rigorous, and reliable.

Qualifications

• Bachelor’s degree or higher in Mathematics, Physics, Chemistry, Engineering, Computer Science, Statistics, Applied Science, or another technical discipline.
• Strong professional proficiency in English, minimum C1, with the ability to write precise technical explanations and evaluation criteria.
• 3+ years of professional, academic, research, or industry experience in your stated technical domain.
• Demonstrated advanced expertise in a clearly defined technical field or specialization.
• Ability to create challenging technical questions that require multi-step reasoning, domain expertise, or professional judgment.
• Strong ability to design detailed evaluation rubrics defining required reasoning, correct methodology, acceptable alternatives, partial-credit criteria, and critical errors.
• Ability to distinguish between a correct final answer and an answer supported by valid versus flawed reasoning.
• Comfortable reviewing and critiquing other experts’ rubrics for ambiguity, missing criteria, redundancy, technical inaccuracies, or poor scoring design.
• High attention to detail when evaluating formulas, assumptions, units, methodology, edge cases, logical consistency, and technical terminology.
• Experience with exam writing, academic grading, peer review, research review, technical QA, standards development, or assessment design is strongly preferred.
• Prior experience with AI evaluation, RLHF, SFT, benchmarking, data annotation, prompt design, or LLM evaluation is preferred.
• Reliable, self-directed, and able to deliver consistent quality in an hourly, remote contractor workflow.

Key Responsibilities

• Create expert-level technical questions: Develop difficult questions that test genuine technical reasoning and expose weaknesses in advanced AI systems.
• Design evaluation rubrics: Write structured criteria defining what a correct, rigorous, and complete response must contain.
• Define partial-credit standards: Identify essential reasoning steps, acceptable alternatives, minor errors, and critical failures.
• Evaluate AI-generated solutions: Assess outputs for correctness, reasoning quality, completeness, methodology, and technical precision.
• Identify hidden reasoning errors: Detect cases where an answer reaches the correct conclusion using invalid assumptions or flawed logic.
• Critique peer rubrics: Review other experts’ criteria for technical accuracy, clarity, completeness, and scoring consistency.
• Develop reference content: Produce expert answers, explanations, critiques, and other gold-standard content.
• Support AI training: Create questions, rubrics, evaluations, preference judgments, and reference-answer pairs suitable for RL and SFT workflows.
• Maintain technical rigor: Ensure content reflects appropriate professional or academic standards within the relevant discipline.