[Hiring] AI Quality Lead @Alight Solutions

Role Description

As the AI Quality Lead for the Artificial Intelligence Center of Excellence (AI CoE), you will architect and drive the end-to-end AI Quality program from inception. This is a hands-on leadership role, requiring both strategic vision and technical depth to go from 0→1 in AI Quality at Alight.

Responsibilities

• Design and implement the unified AI Quality strategy and SOP, scaling controls by risk tier (Tier 1–3)
• Define and own success metrics, risk tiering, acceptance criteria, and operating thresholds for AI solutions
• Lead the development of golden/adversarial test suites (prompt injection, misuse, safety, bias, functional automation)
• Oversee the creation and maintenance of schema-level UI/API tests, agent contracts, and observability instrumentation
• Own the org-wide evaluation harnesses (e.g., promptfoo, TruLens, Ragas), CI gating, and red-team playbooks
• Manage independent spot checks, factsheets, registry entries, and evidence for governance and audit
• Collaborate with Product Owners, Developers, SRE/Observability, and Governance to ensure compliance and operational excellence
• Drive continuous improvement in fairness, explainability, safety, security, and operational SLOs
• Mentor and grow the AI Quality team, including onboarding and coaching junior engineers

Qualifications

• Experience with LLM, RAG, agentic, or classic ML systems
• Familiarity with test automation frameworks, CI/CD pipelines, and evaluation tools (promptfoo, TruLens, Ragas)
• Knowledge of fairness, explainability, safety, and security standards (e.g., OWASP Top 10, MITRE ATLAS, Fairlearn, AIF360)
• Experience with data integrity, lineage, and observability (OpenTelemetry, New Relic, OpenLineage)
• Familiarity with AI infrastructure, orchestration, data integrity, deployments, and model registry practices
• Knowledge of international standards and regulatory frameworks for Responsible AI

Requirements

• Experience leading or building quality engineering or testing programs (AI/ML experience is a plus, but not required)
• Ability to design and implement test strategies, automation frameworks, and quality standards
• Excellent communication and leadership skills; ability to work cross-functionally and influence stakeholders
• Bachelor’s or Master’s degree in Computer Science, Engineering, Data Science, or related field, or equivalent experience

Benefits

• Programs and plans for a healthy mind, body, wallet, and life
• Variety of health coverage options
• Wellbeing and support programs
• Retirement plans
• Vacation and sick leave
• Maternity, paternity & adoption leave
• Continuing education and training
• Voluntary benefit options