[Hiring] Assessment Psychometrist @Pearl

Role Description

We're hiring a Psychometrician for Pearl, which finds exceptional talent from around the world, trains them to be AI-native, and places them into operational roles at startups as managed contractors. You'll be the scientific backbone of our AI voice-model initiative, defining what we should measure, designing and validating the instruments that measure it, and converting those measurements into defensible ground-truth labels and evaluation criteria for model training.

What You'll Own

• The Psychometric Framework
  • Review literature and evaluate established frameworks (Big Five/OCEAN, HEXACO, DISC, etc.) for validity, usefulness, and fit for our use case.
  • Recommend constructs and dimensions to include, exclude, or treat cautiously.
  • Build and maintain a clear construct map connecting constructs, dimensions, indicators, survey items, and resulting scores.
• Instrument Design and Validation
  • Design psychometric surveys that serve as a high-quality measurement source for participants providing voice samples.
  • Decide when to use validated existing scales, adapt them, or develop fit-for-purpose measures.
  • Run the full validation battery: reliability, EFA and CFA, construct/convergent/discriminant/criterion validity, test-retest, and IRT.
  • Recommend sample sizes, pilot methodology, and evidence thresholds for measures.
• Ground Truth and Model Evaluation
  • Partner with AI and Data teams to transform psychometric responses into defensible training labels.
  • Define rules for missing, inconsistent, or low-quality responses.
  • Design the framework for comparing survey-based ground truth against voice-model predictions.
  • Analyze model performance and prediction quality across constructs and populations.
• Bias, Fairness, and Scientific Guardrails
  • Evaluate measurement invariance and potential bias across languages, cultures, and populations.
  • Define scientific limits around conclusions drawn from voice-based predictions.
  • Translate complex statistical findings into practical decisions for stakeholders.

What Success Looks Like

Within your first cycle, the full measurement pipeline runs through you: framework selected, construct map built, Survey V1 designed and piloted, statistically validated, refined into Survey V2, and converted into a scoring system that produces training labels the AI team trusts.

Qualifications

• Advanced degree (MSc or PhD preferred) in Psychometrics, Quantitative Psychology, Psychological Measurement, I/O Psychology, Behavioral Science, or a closely related quantitative field.
• Hands-on instrument experience in designing, validating, and refining psychometric instruments.
• Strong statistical foundation in factor analysis, reliability, validity, and measurement theory; IRT experience is a strong plus.
• Comfort with messy, real-world data and making evidence-based recommendations.
• Cross-functional fluency with AI/ML engineers, data engineers, and product teams; Python or R experience is highly desirable.
• Scientific backbone to challenge unsupported assumptions and define responsible limits for AI-based psychological inference.

Benefits

• Build and Grow Quickly - Opportunity to step into leadership early and own decisions that shape company growth.
• Fully Remote - Work with a multicultural team across 23 countries.
• Unlimited PTO - Space for personal time away with loved ones.
• Global Retreats - Opportunities to connect with teammates in various locations.
• Ambitious and Kind Team - A low-ego, no-assholes policy with a focus on competence and kindness.