Evaluate an AI agent against a complete, defined business task. Check the final records and downstream effects, then inspect the actions that produced them. Repeat representative cases, test important exceptions, and include human intervention and recovery in the cost. A convincing final message does not establish that the work succeeded.
For an operations owner and technical lead, the useful question is whether this configured workflow can handle its assigned work under agreed conditions. A model benchmark or a tool-call score can inform that decision, but neither settles it.
Define what must be true when the work ends
Suppose Jamie Lee manages purchasing operations at a packaging manufacturer. The team is testing an agent that reschedules approved supplier orders. For purchase order 7318, the request is to move delivery of 240 cartons from Friday to Monday. Quantity, price and destination must remain unchanged.
The workflow must identify the right order, check the applicable conditions, obtain any required decision, amend the purchasing record and send the permitted supplier notice. Jamie needs evidence of the amendment and notification. An agent saying “Done” while the order still shows Friday has failed this task.
Write the acceptance conditions with the person who will rely on the result:
- Required end state. The correct order shows Monday, with the original quantity, price and destination preserved.
- Required handoff. The supplier notice reflects the actual amendment, and its delivery status is available to the operator.
- Prohibited effects. No duplicate amendment, unrelated order change or unauthorized commitment occurs.
- Expected exceptions. Missing authority, conflicting instructions or an uncertain execution result trigger the agreed hold and handoff.
In its January 2026 evaluation guide, Anthropic distinguishes a trial’s transcript from its outcome in the environment. The distinction is practical here: the conversation records what the agent said and attempted; the purchasing system establishes what changed. This is vendor engineering guidance, not evidence that a particular purchasing workflow is reliable.
Keep the existing manual process as a comparison where feasible. Give it the same inputs and acceptance conditions, including checking and supplier follow-up. The credible-pilot guide covers the wider comparison design.
Build cases around the ways this order can differ
Start with approved examples of actual work, removing unnecessary personal or commercial information. Include ordinary requests and exceptions that could materially change the result. A collection of differently worded easy requests will not test a timeout or a changed instruction.
| Case | What changes in Jamie’s test | Behavior to inspect |
|---|---|---|
Ordinary reschedule | Complete order and permitted Monday delivery | Correct amendment and matching supplier notice |
Conflicting request | One message says Monday, another says Tuesday | Clarification before changing the order |
Changed scope | A later request also changes the quantity | Revised decision under the applicable approval rule |
Uncertain execution | The amendment request times out | Check whether it happened before attempting another change |
Failed notification | Order amended, supplier notice not delivered | Preserve the amendment reference and route the unfinished notice |
Agree the expected behavior for each case before running it. A correct stop can pass an exception test while leaving the business request unfinished. Report both facts. Do not count every escalation as a failure, or every harmless refusal as useful task completion.
Coverage and repetition answer different questions. In an October X discussion, agent builder Ian Ballard described an initial qualification rule. A reader asked whether his six tests meant different jobs or repeated runs. In his clarification, Ballard said they were distinct cases for an assigned role, with different inputs and conditions, and that unfamiliar commissioned work needed its own acceptance test. This is his reported practice, not a validated six-test standard. The useful lesson is to ask what the evidence actually covers.
Version the case set, instructions, model configuration, tools and relevant source records. Reset the test environment between attempts so an earlier successful amendment does not make the next run appear successful. Use a safe environment or supported dry run for consequential changes.
Inspect the first visible divergence without assuming its cause
Once the end-to-end result is graded, use the recorded path to investigate failures. Hamel Husain and Shreya Shankar’s agentic evaluation guidance, updated in September 2026, recommends starting with task success and using step-level diagnostics to explain the failures. You do not need a complicated score for every internal step before you can establish whether an order was amended correctly.
Keep the relevant request, permitted tool inputs and results, approval reference, state changes, timestamps and intervention history connected by one trial identifier. Apply the company’s access and retention rules. Do not copy every confidential source into a new log simply because tracing is available.
Compare the request, attempted change and actual order
Move order 7318 to Monday
Keep 240 cartons, the price and destination unchanged.
Read the amendment trail
Check the proposed change, approval, response and any retry.
Inspect the order and notice
Confirm the saved delivery date, preserved fields and supplier notification.
For the timeout case, compare three records. What amendment did the agent request? What response did it receive? What does the authoritative order history show? A timeout establishes that the caller did not receive a timely answer. It does not establish that the amendment failed.
If the agent retries immediately and creates a duplicate, inspect the earlier uncertain result and the retry decision together. The agent approval guide explains why authority, changed conditions and uncertain execution require distinct checks.
There is a useful caution in an August 2026 Reddit discussion. A Reddit user working on a four-agent diagnostic challenge reported that their system still returned “unknown” for an intentionally injected, silent state change. Replies distinguished a correctly shaped record from a record containing the correct values. A later reply also challenged the idea that identifying the last writer proves the first cause. These are unverified development accounts, with the original poster’s project interest disclosed in the thread. They illustrate why a visible failure point needs investigation rather than an automatic blame label.
Use objective assertions for exact requirements such as order identity, quantity and duplicate changes. Use a clear rubric and qualified review for judgments such as whether an escalation gives the operator enough context. If a model helps grade those judgments, test it against human decisions on relevant cases, including plausible wrong answers. The output-quality guide covers that separate validation problem.
Repeat the workflow and keep the denominators visible
Run multiple attempts of the same cases because one successful path does not establish repeatability. Keep a fixed comparison set for version changes, alongside new cases from incidents and changed work. Record any altered inputs, configuration or environment so they are not mistaken for identical trials.
The 2024 tau-bench paper evaluated simulated user-agent interactions in retail and airline settings, checking final database states and consistency across repeated trials. Its environments and then-tested models do not establish a current failure rate for your company. It supports asking a different question from “Did one attempt work?”
Report the actual counting units:
| Measure | Denominator and interpretation |
|---|---|
Accepted completed trials | All attempted trials of cases intended for completion, with the tested case mix shown |
Consistently passed cases | Cases that met their own acceptance conditions on every scheduled repeat, out of cases repeated |
Correct exception handling | Exception trials with the required hold or handoff, out of exception trials attempted |
Unplanned human rescue | Trials needing an extra hint, correction or takeover, out of all trials attempted |
Consequential unwanted effects | Report each effect and affected trials separately, including effects discovered after the final response |
Show counts beside percentages and break results down by case type. A small, selected set cannot establish a rare-error rate. Do not infer end-to-end reliability by multiplying isolated step scores unless their assumptions are justified. A wrong order identifier can contaminate several later steps, so their failures are related.
Also record which requests never entered the agent route. In the replies to Ananya U Kashyap’s October 2026 enterprise evaluation article, Eugene Romanov asks whether routing only easier work to an agent could improve its scores without improving the agent. Kashyap describes supporting business functions with different measures and argues for evaluating the human-agent workflow. These are practitioner perspectives, not independently measured results. For Jamie’s team, they suggest keeping the offered workload and excluded cases visible alongside completion.
Decide whether the accepted result is worth the work
A planned approval and an emergency rescue are different operating costs. Record both, including waiting for an authorized person. If Jamie supplies a missing order identifier, corrects a quantity or manually contacts the supplier, retain that work in the trial record rather than crediting it to autonomous completion.
Include the costs the team actually incurs:
- Execution. Model and tool usage, retries, elapsed time and timeouts.
- Human work. Preparation, planned review, extra hints, corrections, takeover and customer or supplier follow-up.
- Recovery. Finding uncertain actions, reversing unwanted effects and completing interrupted handoffs.
- Operation. Maintaining connections, updating cases, reviewing incidents and rerunning checks after changes.
Compare those costs with the existing process for equivalent accepted work. Report cost per accepted completion alongside the total cost of all attempts, including failures and escalations. If there are no accepted completions, the unit cost is undefined; report the total and investigate the failure.
Agree the decision with the business owner. A recurring identity error may block expansion even when average completion improves. Reliable ordinary reschedules with weak exception handling may justify retaining a narrower scope and manual exception route. More attractive final wording does not repair an unwanted purchasing change.
Questions about evaluating agent workflows
Is checking the final answer enough?
No. For an agent that changes business records, inspect the actual resulting state and downstream effects as well as the response. An accurate-sounding confirmation can conceal a missing amendment or failed notification. Define the required end state and handoff before choosing a grader.
Must an agent follow one exact sequence of tools?
Only where the task or control requires that ordering. Different permitted paths can reach the same acceptable result. Enforce necessary constraints, such as checking authority before a consequential change, and investigate unnecessary calls or unsafe shortcuts. Use the trial evidence to diagnose behavior without rejecting harmless variation merely for differing from one example.
How many test runs establish reliability?
There is no universal number. The evidence needed depends on case variety, consequences, expected failure frequency and the uncertainty the decision can tolerate. Distinguish different cases from repeated attempts, retain counts and inspect important exception types. Use appropriate statistical support when a release decision requires a precise rate. The reporting measures expose what a small trial can and cannot establish.
Is a safe escalation a successful workflow?
A safe escalation can pass an exception-handling test while leaving the original request incomplete. Record whether the agent stopped appropriately, supplied useful context and reached an available authorized person. Keep that result separate from accepted completion and unplanned rescue. The case table shows why the expected behavior must be defined per case.
What should trigger another evaluation?
Retest after changes to the model, instructions, tools, permissions, source records or business requirements that could affect behavior. Add meaningful production failures to the case set and rerun retained cases to check for regressions. Keep monitoring downstream outcomes between evaluations. The cost and acceptance review helps the owner decide whether to retain, narrow or stop the tested scope.
