[Hiring] Data Science & Analytic Methodology SME @YO AI Labs

Role Description

We are seeking experienced Data Science & Analytic Methodology Subject Matter Experts to contribute to a high-impact project focused on advanced analytical pipelines and large language models (LLMs). In this role, you will apply your domain expertise to:

• Evaluate AI-generated outputs
• Assess analytical workflows
• Help improve the accuracy, reliability, and performance of next-generation AI systems

Prior experience in AI is not required, although relevant technical experience is highly valued.

Qualifications

• Demonstrated experience integrating large language models (LLMs), including government-specific GPT variants, into complex data or analytical pipelines.
• Proven experience conducting analysis at scale, with a strong understanding of automation failure modes and relevant evaluation metrics.
• Experience in teaching, mentoring, or providing technical leadership in data science or related analytical disciplines.
• Strong technical writing skills, with the ability to document methodologies clearly and communicate complex concepts in English.
• Familiarity with designing evaluation frameworks, benchmarks, or assessment systems.
• Experience in defense, intelligence, or government analytical environments is highly valued.
• Ability to work effectively in distributed, international project environments with minimal clearance-related constraints.

Requirements

• Map large-scale analytical and pipeline integration scenarios to established taxonomies across key evaluation areas.
• Develop detailed analytical cases reflecting real-world deployment patterns, including automated triage, data enrichment, analysis at scale, and common operational failure modes.
• Evaluate and grade responses based on source handling, reasoning quality, understanding of limitations, and practical utility.
• Identify and flag instances of model over-refusal that may disrupt legitimate analytical workflows or pipeline continuity.
• Document complex edge cases, emerging methodological challenges, and opportunities for improvement, providing actionable recommendations.
• Deliver clear, insightful written and verbal feedback to support iterative development and high-quality project outcomes.

Ideal Candidate Profile

The ideal candidate combines strong data science expertise with practical experience in analytical methodology, automation, and LLM integration. They can:

• Assess complex workflows
• Identify limitations and failure patterns
• Evaluate model performance
• Provide actionable feedback to improve the reliability and effectiveness of AI-driven analytical systems.