Next, test the decisions that sit between reliable code checks and final human review. An agent can call the right tool but still choose the wrong sequence, retrieve an outdated policy, or carry a detail from one task into another when it should not.
Build scenario tests around the moments where context changes the outcome. For a support agent, test whether it checks account status before suggesting a refund. For an operations agent, test whether it confirms missing inputs before creating a report rather than filling gaps with guesses.
Cover the full workflow, not only the final reply. A useful set of scenarios can check:
Planning: does the agent break a three-step task into a sensible order
Tool selection: does it choose an approved knowledge source instead of searching unrelated records
Retrieval quality: does it pull the current document and cite the relevant passage
Memory use: does it retain a stated preference during one task without carrying it into a different customer case
Multi-step completion: does it verify each tool result before moving to the next action
If you do one thing, start with the five workflows that handle the highest volume, money, customer data, or approval risk. A scenario that takes 10 minutes to write can catch a repeated error before it reaches hundreds of real interactions.
But do not grade every test against one exact sentence. Model wording can vary while the work is still correct, so define observable success criteria such as whether the agent asked for a missing order number, selected an allowed tool, flagged uncertainty, and stopped before an unapproved action.
A common mistake is scoring only a polished final response. Fix this by recording the plan, retrieved sources, tool calls, and completion state, then checking each against the scenario rules. This works best for repeatable workflows; it fails when success depends mainly on tone or a judgment call that requires a person.
So, add adversarial cases before users create them for you. Give the agent conflicting instructions, incomplete records, an unavailable tool, or a request that should be refused, and check that it identifies the problem instead of forcing a result.
For example, a finance assistant receiving “send the payment now” after a later message says “wait for approval” should pause and request a clear decision. When a required lookup tool is unavailable, the right outcome may be a short status note and escalation, not an invented answer.