Job Description
About the Role
Testing AI systems is a fundamentally different problem than testing traditional software. Outputs are non-deterministic. Correct is often a spectrum. And the failure modes—hallucinations, drift, prompt injection—don't show up in unit tests. We need an engineer who understands this and can build the testing strategies, evaluation frameworks, and quality infrastructure to keep our agents reliable in production.
As an AI Quality Engineer, you'll design how we test intelligent agents, agentic workflows, and Foundation Layer capabilities. This is not a manual QA role—you'll write code, build evaluation pipelines, and create automated testing frameworks that run in CI/CD. You'll define what quality means for AI systems at AGS and build the systems to measure it.
You'll work across every solution the team builds, which means you'll have broad visibility into the architecture and deep understanding of how our agents...