[Hiring] Software Test Engineer @Bigbear.ai

Role Description

Ask Sage, a BigBear.ai company, is hiring a Software Test Engineer to strengthen automated validation across our AI platform. This is not a manual QA role, and it is not a DevSec role. We are looking for a software engineer who specializes in quality engineering: someone who can understand backend code, reason through distributed system behavior, write high-quality automated tests, and improve the infrastructure that gives engineers reliable release signal.

You will partner with engineers to define acceptance tests that prove a fix works and fail reliably if the fix is reverted, and your work will directly improve release velocity, platform reliability, and confidence in complex AI workflows.

What you will do

• Develop and maintain smoke tests, unit tests, integration tests, and end-to-end tests across backend and platform workflows.
• Validate backend changes for new PRs through automated test coverage, direct API testing, and user-facing workflow verification where appropriate.
• Exercise backend behavior through the UI when useful, without owning visual design, frontend UX validation, or manual UI regression testing.
• Build regression coverage for edge cases, malformed inputs, authorization boundaries, concurrency issues, failure modes, and other non-happy-path scenarios.
• Improve the scalability, determinism, execution performance, maintainability, and diagnostic quality of the test suite.
• Strengthen test fixtures, mocks, test data management, failure analysis, and CI feedback loops.
• Design end-to-end tests for AI platform workflows while minimizing unnecessary token usage, external provider calls, latency, and test cost.
• Apply AI-assisted development and analysis tools to accelerate test design, test generation, triage, and maintenance while preserving reliability, reviewability, performance, and deterministic validation standards.
• Validate backend APIs, authentication flows, billing and token behavior, model routing, AI workflow execution, file parsing, MCP/tool execution, and passthrough APIs.
• Validate AI platform E2E paths involving prompts, model responses, streaming behavior, tool calls, agents, workflow orchestration, and provider-facing API compatibility.
• Validate agentic harnesses, MCP integrations, and workflow automation systems, including concepts common to no-code and low-code workflow builders such as Power Automate, Zapier, Make, n8n, and similar platforms.
• Validate security-sensitive product behavior such as user isolation, permission boundaries, validation, sanitization, rate limits, replay prevention, safe error handling, and layered control behavior.
• Build and maintain test infrastructure that must scale with a growing platform, expanding product surface area, and active engineering team.

Qualifications

• 8-10 years of experience with software testing.
• Strong software engineering background with deep experience in automated test development.
• Experience testing backend services, APIs, distributed systems, and database-backed applications.
• Experience designing smoke, unit, integration, and end-to-end testing strategies.
• Strong judgment around edge cases, non-happy paths, adversarial inputs, regression risk, and failure isolation.
• Experience improving CI test reliability, execution time, parallelization, test isolation, and failure observability.
• Familiarity with testing Defense in Depth behavior: validating that multiple layers of checks work together, without this being a dedicated DevSec role.
• Strong troubleshooting, analytical, and communication skills.
• Ability to work independently and as part of a team.
• Ability to obtain clearance.

What we'd like you to have

• Experience writing E2E tests for AI platforms, LLM applications, model gateways, agents, MCP tools, or tool-using systems.
• Experience designing AI E2E tests that control token usage, external provider calls, latency, and cost.
• Experience testing workflow automation systems, agentic harnesses, or no-code/low-code automation platforms (e.g., Power Automate, Zapier, Make, n8n).
• Familiarity with flagship Generative AI provider APIs (Google VertexAI, AWS Bedrock, Microsoft Azure OpenAI) and models (OpenAI GPT, Anthropic Claude, Google Gemini).
• Experience with CI/CD pipelines (e.g., GitHub Actions), Docker, Kubernetes, and observability/monitoring tooling.
• Experience with PostgreSQL and SQL for validating stored data.
• Knowledge of government compliance frameworks (FedRAMP, NIST AI RMF, CMMC 2.0).

Pay transparency

Please note the targeted compensation range is provided as an estimate, and any actual compensation offer may vary depending on the needs of the company, or an applicant's skillset, competencies, experience, education, certifications, location, or other factors. The estimated range does not include the value of any benefits offered.