[Hiring] Staff Applied AI Product Engineer @Carrum Health

Role Description

As a Staff Applied AI Product Engineer, you will bridge the gap between cutting-edge AI capabilities and real-world patient needs. You will take a hands-on leadership role in architecting and integrating LLM-driven features into our core platform. While our foundation is built on Ruby on Rails (SOA) and React, you will lead the evolution of our stack to support AI-native workflows and build agents to support key business initiatives such as improving care navigation quality and efficiency.

• Work closely with Data and Product teams (including a dedicated Applied AI PM) to determine use cases that require LLMs rather than ML.
• Build robust RAG (Retrieval-Augmented Generation) pipelines, select appropriate foundation models, iterate on prompts, and design the "glue" that turns raw model intelligence into reliable product features.
• Partner with our DevOps team to leverage AWS/Azure for simple and cost-effective cloud infrastructure deployment.
• Implement comprehensive monitoring and observability practices to guarantee the high availability, security, and scalability of our new AI services.
• Collaborate directly with our internal clinical experts to validate model outputs, ensuring your AI agents are safe, accurate, and truly helpful in a healthcare setting.
• Own the AI engineering strategy, moving us beyond "demos" into scalable, production-ready systems.
• Establish patterns for building “smart products,” model evaluation, latency optimization, and cost management.
• Champion Responsible AI by operationalizing the internal guidelines set by our AI Council.
• Mentor the wider engineering team on AI-native development practices (such as prompt engineering and context management).
• Collaborate with product managers to identify high-leverage opportunities where AI can radically simplify the healthcare experience.
• Partner with the VP of Engineering and Chief Product Officer to staff and align delivery teams on your initiatives.
• Define project timelines and milestones from inception to go-live.

Qualifications

• 10+ years of software development experience.
• 2+ years of dedicated professional experience architecting production-grade applications with LLMs (OpenAI, Anthropic, Llama).
• Experience working in a web application environment (Ruby on Rails, Python).
• Comfortable working with DevOps to design and configure Terraform for AI infrastructure.
• Experience handling deadlines and prioritizing work in a fast-paced, dynamic environment.
• Strong interpersonal skills and empathy.

Requirements

• Expertise in building robust, scalable, high availability systems.
• Knowledge of RAG pipelines and vector databases (e.g., pgvector, Pinecone).
• Ability to leverage prebuilt solutions versus custom LLM integrations.
• Experience in exposing AI logic via performant APIs to frontend clients (React).
• Drive execution and elevate the team through technical leadership.
• Use AI developer productivity tools (e.g., Cursor, GitHub Copilot) daily.
• Understanding of latency, error handling, monitoring/alerting, and user experience.
• Comfortable with SQL and data modeling.
• Entrepreneurial mindset, excelling at going from 0 to 1.

Benefits

• Stock option plan.
• Flexible schedules and remote work.
• Chicago and San Francisco offices available.
• Self-managed vacation days, within reason.
• Paid parental leave.
• Health, vision, and dental insurance.
• 401K retirement plan.