[Hiring] Forward Deployed Machine Learning Engineer @Advatix

Role Description

Our Client is seeking a Forward Deployed Machine Learning Engineer (FDE MLE) with strong experience in machine learning evaluation, benchmark systems, backend infrastructure, and customer-facing engineering. This individual will be the first Machine Learning Engineer dedicated to the organization's Benchmarks and Evaluations vertical and will work closely with the General Manager, researchers, and early customers.

The successful candidate will help establish the technical foundation for evaluating AI models across different domains and modalities. This is a highly hands-on role combining machine learning engineering, backend infrastructure, data pipelines, evaluation systems, and customer-facing technical delivery.

The ideal candidate is comfortable operating in ambiguous, fast-moving environments and can independently take technical problems from feasibility through production delivery. Strong customer engagement skills and experience owning technical projects end-to-end are essential.

Key Responsibilities

• Partner directly with the General Manager, researchers, and early customers to define, design, and build AI benchmarks and evaluation systems.
• Develop benchmarks and evaluation frameworks across multiple AI domains and modalities.
• Build and own backend infrastructure supporting AI model evaluation.
• Design and maintain data pipelines, execution environments, storage systems, and orchestration infrastructure.
• Build sandboxed environments for agentic evaluations involving tools, code execution, and multi-step tasks.
• Develop and deploy end-to-end evaluation systems for foundation models.
• Own the engineering component of customer engagements from initial requirements through technical delivery.
• Work directly with enterprise customers to understand technical requirements and develop practical evaluation solutions.
• Identify repeatable evaluation patterns that can be transformed into scalable infrastructure and product capabilities.
• Identify infrastructure gaps and opportunities that can inform future product development.
• Scope ambiguous technical problems, assess feasibility, and independently drive solutions through implementation and delivery.
• Collaborate with AI researchers and other technical stakeholders to translate research concepts into reliable production systems.
• Develop scalable systems capable of supporting large-scale machine learning evaluation and benchmark workloads.
• Communicate technical concepts, project status, tradeoffs, and solutions clearly to customers and internal stakeholders.

Qualifications

• Minimum 4+ years of professional engineering experience with hands-on machine learning model evaluation experience.
• Minimum 3–8 years of relevant experience across machine learning engineering, evaluation systems, benchmark development, and end-to-end technical ownership.
• Demonstrated experience deploying end-to-end ML evaluation or benchmark systems to production for foundation models.
• Strong hands-on experience with ML evaluation frameworks and benchmark design, including approaches such as LLM-as-a-judge.
• Prior ownership of backend and infrastructure systems, including:
  • Data pipelines
  • Execution environments
  • Storage
  • Orchestration
• Experience building benchmarks, evaluations, or human-data pipelines for large language models (LLMs) is strongly preferred.
• Experience building data pipelines capable of supporting large-scale workloads.
• Experience working with or deploying machine learning systems in production.
• Customer-facing engineering experience, including direct interaction with enterprise customers.
• Demonstrated ability to independently scope and execute ambiguous technical problems from feasibility through delivery.
• Strong written communication skills for customer-facing and cross-functional technical work.
• Strong bias toward action and ability to operate effectively in fast-moving, high-ambiguity environments.
• Bachelor's degree or higher in Computer Science, Physics, or a related technical field.

Required Experience & Environment

• Candidates should demonstrate experience in at least one of the following types of environments:
  • Early-stage B2B startups with demonstrated traction.
  • Forward-deployed engineering organizations or companies with a strong FDE model.
  • High-ownership generalist roles within larger organizations, such as an Office of the CTO or internal machine learning evaluation team.
  • Fast-moving engineering environments where individuals have owned projects from initial concept through production deployment.

Customer-Facing Engineering Expectations

• The ideal candidate should be comfortable:
  • Working directly with enterprise customers.
  • Translating customer requirements into technical specifications.
  • Managing technical components of customer engagements.
  • Explaining complex ML evaluation concepts clearly to technical and non-technical stakeholders.
  • Balancing customer-specific requirements with the development of reusable infrastructure.
  • Working under tight customer deadlines while maintaining engineering quality.
  • Taking ownership of technical delivery from initial discovery through deployment.

Preferred Qualifications

• Experience working directly with AI researchers or foundation model labs.
• Experience developing evaluation systems for foundation models or LLM-based applications.
• Published research, papers, or open-source contributions related to ML evaluations or benchmarks.
• GitHub contributions involving ML evaluation, benchmark systems, LLM evaluation, or related infrastructure.
• Experience with agentic AI evaluation environments involving tools, code execution, or multi-step workflows.
• Experience developing human-data pipelines for machine learning evaluation.
• Experience working across multiple AI modalities or evaluation domains.

Role Expectations

• The successful candidate should demonstrate:
  • High ambiguity tolerance: Comfortable solving problems where requirements and approaches are not fully defined.
  • Bias to action: Able to move quickly from concept and feasibility assessment to implementation.
  • End-to-end ownership: Takes responsibility for technical outcomes rather than isolated engineering tasks.
  • Customer orientation: Comfortable working directly with enterprise customers and incorporating their requirements into technical solutions.
  • Infrastructure depth: Capable of building the backend systems required to support production ML evaluations.
  • Evaluation expertise: Strong understanding of benchmark design, evaluation methodology, and ML evaluation frameworks.
  • Strong communication: Able to document and communicate technical decisions clearly.
  • Generalist mindset: Comfortable moving between ML evaluation, backend infrastructure, data pipelines, and customer-facing technical work.

Backgrounds Less Aligned With This Role

• This position requires both ML evaluation expertise and strong production engineering/customer-facing experience. Candidates may be less aligned when their background is primarily:
  • Research-oriented MLE work without meaningful production deployment or engineering ownership.
  • Machine learning research without experience building and deploying evaluation infrastructure.
  • Long-term product engineering experience without demonstrated experience in rapid prototyping or highly ambiguous environments.
  • Experience limited exclusively to stealth startups or small B2C companies without relevant B2B, enterprise, or customer-facing engineering experience.
  • ML engineering experience without hands-on benchmark or evaluation system development.
  • Engineering experience without direct customer-facing responsibilities where customer engagement is a significant part of the role.