Job Description
Responsibilities
• Define and execute comprehensive test strategies covering statistical, ML, LLM and agentic AI models.
• Perform functional, regression and scenario‑based testing of model behaviours and workflows.
• Conduct AI/ML evaluations including accuracy checks, bias/fairness assessment, robustness analysis and drift detection.
• Assess end‑to‑end model workflows including data inputs, feature transformations, task completion, tool‑use accuracy and multi‑step reasoning.
• Design and maintain automated test and evaluation pipelines, including benchmarking and regression frameworks.
• Validate API and tool‑integration behaviour in production‑like environments, identifying dependency or orchestration issues.
• Diagnose issues using observability, logging, tracing and debugging tooling, and document findings clearly.
• Collaborate with data scientists across departments to understand modelling intent, f...