[Hiring] Software Engineer, AI Quality & Evaluation @Bask Health

Role Description

We're looking for a Software Engineer, AI Quality & Evaluation to build the engineering systems that help Bask Health test, evaluate, and improve AI-powered applications throughout their development and production lifecycles.

Bask Health supports direct-to-consumer telehealth businesses through a platform that connects patient experiences, merchant operations, clinical workflows, pharmacy coordination, and commerce tools. As we introduce more AI-powered capabilities across these areas, we need reliable software infrastructure that makes it easy for engineering teams to understand how those systems behave.

You'll help develop the internal tools, backend services, testing frameworks, and automation that support AI quality assurance across the platform.

This is a hands-on software engineering role focused on building reusable systems rather than evaluating individual models in isolation. You'll work with APIs, databases, automated testing infrastructure, model integrations, and developer tools to make AI evaluation more efficient, consistent, and accessible.

Your work may include:

• Evaluation execution services
• Test dataset management tools
• Automated regression pipelines
• Result comparison systems
• Interfaces that help engineers investigate AI failures

You'll collaborate with applied AI engineers, machine learning engineers, AI evaluation specialists, and backend engineers to understand their testing needs and translate those requirements into dependable software capabilities.

You'll also help integrate AI evaluation into normal engineering workflows. This includes:

• Making it easier to run tests during development
• Comparing model or prompt changes
• Inspecting failures
• Identifying regressions before production releases

Because Bask operates in healthcare-related environments, you'll help ensure evaluation systems handle sensitive information appropriately, respect access permissions, and support testing for privacy, safety, and approved operational boundaries.

This position emphasizes strong software engineering fundamentals, developer experience, and production reliability. You'll contribute to technical architecture, write maintainable code, improve infrastructure performance, and own services that other engineering teams rely on.

We're looking for an engineer who enjoys building tools for other developers, understands the challenges of testing nondeterministic AI systems, and wants to create reliable infrastructure that improves software quality across the organization.

Qualifications

• 3+ years of professional software engineering experience, preferably in backend, platform, infrastructure, or developer tooling roles.
• Experience building and maintaining production backend services.
• Strong programming skills in TypeScript, JavaScript, Python, or comparable languages.
• Experience designing and developing APIs and service integrations.
• Familiarity with automated testing frameworks and software quality practices.
• Understanding of continuous integration and deployment workflows.
• Experience building reusable software components or internal engineering tools.
• Familiarity with large language models and modern AI application development.
• Understanding of the challenges involved in testing nondeterministic AI systems.
• Experience working with relational databases and structured application data.
• Familiarity with asynchronous processing and background job systems.
• Understanding of distributed systems fundamentals and service reliability.
• Experience developing automated testing pipelines.
• Familiarity with cloud-based application deployment.
• Understanding of authentication, authorization, and secure service communication.
• Experience implementing logging, monitoring, and production diagnostics.
• Ability to troubleshoot backend services and infrastructure issues.
• Familiarity with configuration management and software release processes.
• Experience writing maintainable, well-tested production code.
• Understanding of API error handling, retries, and failure recovery.
• Experience working with Git and collaborative engineering workflows.
• Ability to translate technical requirements into reusable engineering systems.
• Strong analytical thinking and problem-solving skills.
• Ability to collaborate effectively with software engineers, AI engineers, and machine learning specialists.
• Ability to independently own engineering projects from technical design through deployment.

Nice to Have

• Experience building infrastructure for AI evaluation or machine learning testing.
• Familiarity with OpenAI, Anthropic, or comparable model APIs.
• Experience with LangSmith, DeepEval, Ragas, promptfoo, or comparable evaluation tools.
• Familiarity with LLM-as-a-judge evaluation approaches.
• Experience building evaluation execution services or automated benchmark systems.
• Familiarity with AI regression testing and prompt comparison workflows.
• Experience developing internal developer platforms or SDKs.
• Experience building testing infrastructure for distributed applications.
• Familiarity with TypeScript and Node.js backend development.
• Experience with React or Next.js for internal tooling interfaces.
• Familiarity with tRPC, Prisma, Drizzle, or comparable development tools.
• Experience working with AWS Lambda, SQS, EventBridge, or related cloud services.
• Familiarity with serverless infrastructure and infrastructure-as-code tools.
• Experience with SST or comparable application deployment frameworks.
• Experience building scalable asynchronous job processing systems.
• Familiarity with embeddings, vector databases, and retrieval-augmented generation.
• Experience implementing model observability or AI request tracing.
• Understanding of multi-tenant SaaS architecture and data isolation.
• Familiarity with HIPAA and healthcare data privacy considerations.
• Experience using AI-assisted development tools such as Cursor or Claude Code.