[Hiring] Machine Learning Engineer, Core Evaluations @Cantina

Role Description

We are seeking an experienced Machine Learning Engineer (MLE) to focus on audio model evaluation, specifically for speech generation and recognition models. This role involves:

• Designing model evaluation pipelines for models in development and production.
• Designing user studies for subjective model evaluations.
• Converting requirements into measurable metrics.
• Designing and developing automated evaluation dashboards to see model performances and compare results.
• Training new models to capture new and different evaluation metrics.
• Communicating with the model team to help design better models based on the evaluation results.
• Communicating with the data team to help decide the type of data necessary to improve model performance.
• Communicating with the product manager to ensure product requirements are correctly measured.
• Helping grow the evaluation team as the founding member.
• Leading the evaluation team in the future.

Qualifications

• Strong experience and intuition for designing metrics that capture model performance.
• Strong experience with designing user studies on Mechanical Turk or similar platforms.
• Strong experience with model training and fine-tuning for model evaluation.
• Strong statistical knowledge and experience to statistically compare evaluation results and make decisions.
• Very strong engineering and programming skills.
• Experience with training ASR, TTS models.
• Experience in ML teams working on large-scale machine learning problems (>3B models with >1m hours of data).