It passed on Friday. That says nothing about Tuesday.
Nothing in your repository changed. The provider shipped a new model snapshot, and an agent that had settled ten thousand invoices the same way starts deciding differently. Run the same cases against every model you allow and compare the outcomes side by side. Every run is a record you can hand to someone who asks.
Everything you can only do when nothing is at stake.
The awkward cases put through the agent’s real tools: the malformed record, the duplicate claim, the request that looks legitimate and isn’t. Adversaries trying to talk it out of its rules. And where there is a voice line, callers who interrupt, go quiet, or talk over the top of it.
Then the same tests run against the real work.
The same assertions at the deploy gate, and inline against the real work: sampled and rate-limited, scored, and used to decide whether a rollout carries on or stops itself.
Changing an agent while someone is mid-sentence.
A new version goes out to a slice of the traffic first, and carries on only if the numbers hold. If they don’t, it goes back on its own, without anyone watching a dashboard.
The verdict is computed from the platform’s own record of what happened. It doesn’t depend on a monitoring stack being installed.
Someone halfway through a conversation, or a run halfway through its work, stays on the version it started on. A rollout is not a reason for one customer to get two different agents.
On shutdown, in-flight work finishes. That matters more when the thing in flight is a payment than when it’s a chat.
Replicas follow the load, so the Monday queue and the Sunday trickle cost differently.
Once is a sample size of one.
The same cases run ten thousand wide, on one set of machines in your cluster.