[Hiring] QA Automation Engineer @Bigbear.ai

Role Description

Ask Sage (BigBearAI company) is seeking a QA Automation Engineer that will translate nuanced product requirements and customer workflows into effective test coverage, build and maintain automation with Playwright, and partner with Product and Engineering to make informed release decisions as the platform evolves. We are looking for an engineer who can reason about complex, nondeterministic AI behavior, design risk-based coverage that reflects real customer impact, and produce automation that engineers trust. The engineer will own the automated coverage and the evidence behind release readiness across browser, application, and backend boundaries, where auditability, security, and government compliance expectations are part of everyday delivery.

What you will do

• Translate requirements into testable outcomes.
• Own automated test coverage.
• Develop a risk-based testing strategy.
• Test conversational AI workflows.
• Evaluate AI response quality.
• Make automation reliable and useful.
• Communicate release readiness.
• Preserve decision history.
• Cover platform and AI infrastructure surfaces.
• Control test cost and execution footprint.
• Automate security-sensitive validation.
• Apply AI-assisted testing tools responsibly.
• Scale the automation footprint.

Qualifications

• 4+ years of experience in QA automation engineering.
• Demonstrated experience building and maintaining automated tests with Playwright.
• Strong coding ability in TypeScript or JavaScript.
• Experience with API testing, CI/CD integration, and test isolation.
• Ability to reason about complex requirements and explore edge cases.
• Clear written and verbal communication skills.
• Experience testing authentication, authorization, and permissions.
• Familiarity with validating Defense in Depth behavior.
• Strong troubleshooting and analytical skills.
• Ability to obtain a Department of Defense Secret clearance.

Requirements

• Experience testing LLM applications, retrieval-augmented generation, or AI agents.
• Experience with Python for API testing and test utilities.
• Experience with enterprise or government platforms and auditable test evidence.
• Experience testing model gateways, MCP tools, or workflow automation platforms.
• Familiarity with flagship Generative AI provider APIs and models.
• Experience with CI/CD pipelines, Docker, Kubernetes, and observability tooling.
• Experience with PostgreSQL and SQL for test data setup and validation.
• Knowledge of government compliance frameworks.
• Active DoD security clearance at the Secret level or above.

Pay Transparency

Please note the targeted compensation range is provided as an estimate, and any actual compensation offer may vary depending on the needs of the company, or an applicant's skillset, competencies, experience, education, certifications, location, or other factors. The estimated range does not include the value of any benefits offered.