[Hiring] Member of Technical Staff, Frontier AI @24-MAG

Role Description

We are sharing a specialised full-time opportunity for experienced technical professionals to operate at the intersection of AI research, machine-learning data systems, evaluation, and real-world model performance. Selected professionals will take ownership of research and evaluation initiatives designed to generate high-quality, defensible research signal and translate that signal into measurable improvements in AI systems. The role combines evaluation design, ML-oriented data development, failure analysis, quality calibration, and close collaboration with researchers, domain experts, and operational teams.

Key Responsibilities

• Research Evaluation & Signal Quality
  • Own research and evaluation initiatives from problem framing through data design, quality calibration, and signal validation.
  • Define rigorous approaches for determining whether experimental results provide reliable and defensible research signal.
  • Analyse model and system failures to identify root causes, edge cases, and opportunities for improvement.
  • Evaluate whether datasets, experiments, and conclusions meet appropriate quality thresholds.
  • Act as a quality gate when signal strength, data integrity, or supporting evidence is insufficient.
• ML-Oriented Data & Evaluation Design
  • Design ML-oriented data systems including task definitions, annotation schemas, rubrics, incentives, and supporting pipelines.
  • Structure data and evaluation workflows around downstream model-performance objectives.
  • Translate ambiguous real-world behaviour into measurable evaluation frameworks and new data categories.
  • Identify gaps in evaluation or dataset coverage and recommend where additional investment or iteration is needed.
  • Develop quality-assurance processes that maintain strong and consistent research standards.
• Failure Analysis & Iterative Model Improvement
  • Investigate model and system behaviour to identify recurring weaknesses and performance limitations.
  • Iterate rapidly on evaluations, datasets, feedback loops, and quality standards.
  • Use experimental findings to guide improvements in model or agent performance.
  • Determine when research directions should be expanded, revised, paused, or discontinued based on evidence.
  • Maintain a systems-level perspective focused on end-to-end AI performance rather than isolated components.
• Research Collaboration & Technical Communication
  • Work closely with researchers, domain experts, operators, and cross-functional teams throughout project kickoff, calibration, and iteration.
  • Communicate research findings, trade-offs, limitations, and signal strength clearly to technical and non-technical stakeholders.
  • Translate research progress into credible narratives grounded in evidence.
  • Support alignment between experimental work and real-world system requirements.
  • Contribute strong technical judgement in ambiguous, high-impact research environments.

Qualifications

• Strong professional judgement regarding research signal quality and whether findings are ready to support broader conclusions.
• Experience designing ML-oriented datasets, evaluation frameworks, annotation systems, rubrics, or QA processes.
• Ability to translate complex and ambiguous real-world system behaviour into structured research and evaluation opportunities.
• Strong ownership mindset and comfort making decisions in uncertain or rapidly evolving environments.
• Excellent written and verbal communication skills.
• Ability to explain technical trade-offs, limitations, evidence quality, and research findings clearly.
• Proven experience working directly with researchers, technical experts, or domain specialists during project calibration and iteration.
• Systems-level understanding of model, agent, or AI-system performance.
• Experience with reinforcement-learning environments, simulators, or feedback-driven training systems is advantageous.
• Experience improving agentic systems or AI systems operating within real-world workflows is beneficial.
• Prior work within applied research or production environments with direct impact on deployed systems is advantageous.
• Experience designing evaluations for complex or real-world tasks is strongly valued.
• Familiarity with expert incentive design or high-stakes technical research programmes is beneficial.

Engagement Details

• Full-time engagement.
• Fully remote.
• Compensation: $600,000–$2,000,000/year.
• Work will span research evaluation, ML-oriented data design, failure analysis, quality calibration, and iterative AI-system improvement.
• Responsibilities may include acting as a quality gate for research claims, datasets, and evaluation results.
• Collaboration will involve researchers, domain experts, operational teams, and other technical stakeholders.
• Project priorities, evaluation frameworks, and research directions may evolve based on experimental findings and system performance.
• Work must be completed without using confidential or proprietary information belonging to any employer, client, institution, or other third party.

Company Description

This opportunity is available through 24-MAG LLC. We connect experienced professionals with remote consulting opportunities across technical, evaluation, and project-based workstreams.

By submitting this application, you acknowledge that your information may be processed by 24-MAG LLC for recruitment and opportunity matching in accordance with our Privacy Policy: https://www.24-mag.com/privacy-policy