[Hiring] QA Automation Lead @Accuity

Role Description

The Quality Assurance (QA) Automation Lead plays a critical role in ensuring the quality, reliability, and scalability of Accuity's AI-driven clinical documentation and revenue cycle solutions. This highly hands-on technical leadership role is responsible for designing and implementing comprehensive test automation frameworks across web applications, APIs, data pipelines, and AI/ML systems. Working closely with data scientists, AI engineers, product teams, and DevOps partners, the QA Automation Lead establishes modern quality engineering and MLOps practices to ensure systems and models are production-ready, compliant, and performant.

Responsibilities

• Automation Strategy and Framework Development:
  • Define and implement enterprise-grade test automation strategies across AI/ML model evaluation, APIs and service layers, data pipelines, and web applications.
  • Design and build scalable, reusable automation frameworks that support continuous delivery and high test coverage.
  • Establish standardized approaches for functional, regression, performance, and data validation testing.
  • Drive adoption of modern testing methodologies, including shift-left testing and test-driven development practices.
• AI and ML Testing and Evaluation:
  • Develop advanced testing frameworks for model validation, including accuracy, precision and recall, drift detection, and reliability monitoring.
  • Implement evaluation approaches for large language models and AI systems, including prompt testing, output validation, and hallucination detection.
  • Build data quality and feature validation pipelines using synthetic and production-like datasets to support robust model testing.
  • Ensure AI systems meet reliability, explainability, and safety standards appropriate for healthcare environments.
• DevOps and MLOps Integration:
  • Partner with DevOps teams to integrate automated testing into CI/CD pipelines and release processes.
  • Contribute to the design and implementation of MLOps frameworks that support continuous integration and deployment of machine learning models.
  • Implement automated validation gates, monitoring, logging, and feedback loops for model promotion and production performance.
  • Ensure alignment between QA automation practices, DevOps workflows, and Azure-based infrastructure.
• Quality Engineering and Delivery Excellence:
  • Establish quality metrics and KPIs, including test coverage, pass rates, defect leakage, and execution time, and build reporting visibility around performance.
  • Lead defect management processes, including triage, root cause analysis, and resolution tracking.
  • Collaborate with Product Owners and Scrum Masters to embed quality practices into Agile workflows.
  • Ensure test coverage aligns with business-critical workflows across clinical documentation and revenue cycle processes.
• Leadership and Mentorship:
  • Serve as a technical leader and mentor within a small QA and engineering team.
  • Establish best practices for automation, code quality, and test design.
  • Conduct code reviews and provide guidance on automation frameworks, tools, and implementation approaches.
  • Influence engineering culture toward a quality-first mindset and continuous improvement.
• Compliance and Healthcare Alignment:
  • Ensure testing frameworks support auditability, traceability, and regulatory compliance requirements.
  • Validate systems against healthcare-specific workflows, including CDI, coding accuracy, and EHR integrations.
  • Incorporate controls for data privacy, security, and model governance into testing and release practices.

Qualifications

• Bachelor's degree in Computer Science, Engineering, or a related field required.
• 7 to 10+ years of experience in software quality assurance and automation.
• Proven experience designing and implementing automation frameworks across web, API, and data systems.
• Strong programming skills in one or more languages, such as Python, C#, JavaScript, or TypeScript.
• Experience with test automation tools such as Selenium, Playwright, Cypress, or PyTest.
• Strong understanding of CI/CD pipelines, DevOps practices, and version control systems.
• Experience with Agile methodologies and tools such as Jira.

Preferred Qualifications

• Experience testing AI and ML systems, including model validation and data pipeline testing.
• Experience working in Azure-based environments and integrating with Azure DevOps.
• Familiarity with MLOps concepts and frameworks.
• Experience in healthcare technology, including CDI, revenue cycle or coding workflows, and EHR systems and integrations.
• Knowledge of regulatory and compliance requirements in healthcare systems.

Core Competencies

• Deep technical expertise in automation and quality engineering.
• Strong problem-solving and analytical skills.
• Ability to operate hands-on while defining strategy and standards.
• Excellent cross-functional collaboration with engineering, product, clinical, and data teams.
• Strong communication skills with the ability to translate technical quality risks into business impact.
• Leadership and mentorship capabilities.

Additional Requirements

• Physical Requirements:
  • The requirements described here are representative of those that must be met by an employee to successfully perform the essential functions of this job with or without reasonable accommodation.
  • Unless otherwise indicated, Accuity positions require interaction with people and technology while either sitting or standing.
  • Employees must be able to communicate via phone, email, etc. and sit for extended periods of time, with or without reasonable accommodations.
  • Physical effort and exposure to physical risk are limited to that of an office role/environment.
• Position and Employment Statement:
  • While this job description is intended to be an accurate reflection of the job requirements, management reserves the right to modify, add or remove duties from a job and to assign other duties as necessary and at any time.
  • All positions at Accuity Delivery Systems, LLC, are at-will employment, and a position description is not a guarantee of a job or of job responsibilities.