AWS researchers introduced the Agent Evaluation Metric (AEM) on September 10, 2026, to find where an AI agent's multi-step conversation first goes wrong. Instead of judging only whether the final task succeeded, it evaluates each response and tool call and records which errors came from earlier turns.
Their example follows an assistant building a sales report. At turn two, the agent passes profit instead of revenue to a tool. Later answers inherit the incorrect data. A task-level score marks the whole report as a failure; AEM identifies turn two as the original mistake and labels the later failures as consequences.
The first version measures correctness through two components: truthfulness, meaning that values agree with the expected result, and completeness, meaning that required information is present. Tool calls also receive checks for wrong actions and missing or extra parameters. The default composite calculation averages passing turns, though the scoring rule can be changed.
AWS demonstrates the method with Amazon Quick Suite and describes how to use it in an agent evaluation workflow built with Strands Agents. The blog provides a framework and illustrative walkthrough, not measured evidence that AEM has improved agents in production. It diagnoses failure paths; it does not repair them.