Role Description
We are seeking an experienced Life Sciences or Medicine expert to lead a pod of trainers building realistic, terminal-based scientific and technical tasks for Terminal Bench Science. You will review every task your trainers produce for scientific correctness, reproducibility, data compliance, and grader reliability, and coach trainers to consistently deliver high-quality, challenging tasks.
Key Responsibilities
-
Review trainer tasks end to end, including task statements, biological and clinical datasets, reference files, pipelines, environments, reference solutions, and automated tests.
-
Verify that tasks reflect authentic research or clinical analysis workflows and require genuine multi-step scientific reasoning.
-
Check that tools, reference genomes, annotation versions, databases, and dependencies are pinned and that pipelines are reproducible.
-
Validate scientific correctness, including genome builds, coordinate conventions, normalization, batch-effect handling, statistical assumptions, multiple-testing correction, units, dosing conventions, imaging orientation and spacing, and random-seed control.
-
Ensure graders verify substantive outputs such as variant calls, differential expression results, cluster assignments, binding or structural metrics, survival estimates, model performance, segmentation accuracy, and pharmacokinetic parameters rather than formatting.
-
Ensure tolerances account for legitimate stochastic and numerical variation without accepting incorrect analyses.
-
Ensure tasks cannot be gamed through hard-coded outputs, reading test files, or bypassing the intended analysis.
-
Confirm that all data is synthetic, public, appropriately licensed, and de-identified, with no protected health information or restricted-access data.
-
Screen tasks for biosafety and dual-use concerns.
-
Escalate content involving dangerous pathogens, toxins, or enhancement of harmful biological capabilities.
-
Provide clear, actionable written feedback and track revisions through closure.
-
Onboard and mentor trainers.
-
Allocate work, track quality and throughput, and escalate blockers.
-
Maintain pod documentation covering guidelines, recurring issues, and exemplary tasks.
Qualifications
-
Ph.D., M.D., M.D.-Ph.D., postdoctoral experience, or equivalent advanced experience in a life science, biomedical, or medical field with a substantial computational component.
-
Strong programming skills in Python or R, plus Bash, with Linux and terminal proficiency.
-
Hands-on experience with bioinformatics pipelines, computational biology, biostatistics, medical imaging analysis, or clinical data analysis.
-
Strong understanding of experimental design, statistical inference, and reproducibility in biomedical research.
-
Proven experience reviewing code, analyses, manuscripts, or research outputs.
-
Experience mentoring or leading small technical or research teams.
-
Excellent written communication skills for providing precise, constructive feedback.
Preferred Qualifications
-
Familiarity with tools such as samtools, bcftools, GATK, BWA, STAR, Salmon, DESeq2, edgeR, Seurat, Scanpy, Biopython, Bioconductor, Nextflow, or Snakemake.
-
Familiarity with RDKit, OpenMM, GROMACS, AutoDock Vina, PyMOL, or Biopython structure tools.
-
Familiarity with medical imaging and clinical tools such as pydicom, nibabel, SimpleITK, MONAI, FSL, lifelines, statsmodels, or FHIR data formats.
-
Familiarity with Docker, Conda, Git, CI systems, automated testing, and HPC environments.
-
Experience designing benchmarks, analysis challenges, or computational evaluations.
Bonus Points
-
Experience evaluating AI coding or terminal agents.
-
Experience with data governance, IRB processes, or biosafety review.
-
Industry experience in biotech, pharma, clinical research, or health technology.
-
Publications or open-source contributions in computational biology, bioinformatics, or biomedical informatics.
-
Prior experience in AI training or evaluation programs.