[Hiring] AI Operations Specialist @Health Note

Role Description

We are looking for a hands-on AI Operations Specialist to own customer AI implementations from discovery through launch and ongoing optimization. You will configure and improve LLM-powered workflows, evaluate production behavior, troubleshoot issues, and work directly with customers and internal teams to make sure implementations work reliably in the real world. This is a customer-facing technical IC role. You should be comfortable moving between AI behavior, implementation details, customer workflows, and production troubleshooting.

What You’ll Do

• Own customer AI implementations from discovery through launch and ongoing optimization.
• Translate customer workflows and requirements into clear AI behavior, implementation plans, and testing criteria.
• Design and improve prompts and workflow logic for customer-facing AI agents.
• Build and run evaluations, test plans, and regression checks to improve reliability and consistency.
• Investigate production issues across prompts, workflows, integrations, data, APIs, and application behavior.
• Determine whether problems can be resolved directly or require product or engineering support, and escalate with clear evidence and reproduction steps.
• Facilitate customer testing and troubleshooting, and communicate status, risks, blockers, and required decisions clearly.
• Drive open issues to resolution across customers and internal teams, including when ownership or next steps are ambiguous.
• Use recurring customer and production issues to improve implementation practices and product behavior.

Qualifications

• Hands-on experience working with LLM-powered products or workflows in production.
• Experience building evaluations and improving unreliable or nondeterministic AI behavior.
• Strong technical troubleshooting skills and the ability to investigate complex systems without immediately handing problems to engineering.
• Experience owning customer implementations from discovery through launch and optimization, including working directly with customers throughout the process.
• Experience with healthcare technology and/or conversational or voice AI is helpful but not required.

Benefits

• Health, dental, and vision insurance
• 401(k) with company match
• Flexible PTO and company-paid holidays
• Fully remote work within US hours
• Company-provided laptop