Method
A single percentage hides the only distinction that matters in operations: mistakes you can absorb, and mistakes that reach a customer.
"It's 94% accurate" is the most common sentence in AI procurement and one of the least useful. Ninety-four percent of what, measured how, and — the question almost nobody asks — what happens in the other six?
Consider two agents handling returns, both scoring 94%. The first one's errors are all over-cautious: it escalates cases it could have decided. Its 6% costs you a few minutes of staff time each. The second one's errors are confident approvals of refunds that policy would have declined. Its 6% costs you money, and eventually a pattern that gets shared on a forum.
Same number, completely different systems. Any evaluation that produces one figure has thrown away the distinction your business actually runs on.
We break every task's errors into four buckets and set a separate tolerance for each:
Written this way, a client can make an actual decision. A system at 88% headline accuracy with 0.2% silent-wrong is frequently a better buy than one at 94% with 3% silent-wrong, and once you show operators the split they reach that conclusion themselves in about a minute.
Thresholds only work if the confidence score means something. A model that says 0.9 should be right about nine times in ten at that score — and out of the box, most are not calibrated at all. We check this directly: bucket the shadow-mode predictions by stated confidence, then plot how often each bucket was actually correct.
The curve is usually wrong in a predictable direction — over-confident in the middle, sensible at the extremes. Fixing it is unglamorous work: recalibration on held-out data, and sometimes replacing a self-reported score with an ensemble signal. But without it, "escalate below 0.8" is a threshold on a number that does not mean what it says.
An evaluation set built once, at the start, decays. Catalogues change, promotions create new edge cases, a marketplace rewrites its category rules, and the set that certified the system in March is quietly irrelevant by August.
So we treat it as a maintained artefact. Every escalation that a human resolves differently from the agent's proposal becomes a candidate test case. Every incident becomes a permanent regression test. The set grows by roughly 5–10% a month for the first year, and it is handed over with the code — of everything we deliver, it is the piece clients most often say they would not have built themselves.
Accuracy in isolation invites a bad trade. It is nearly always possible to raise a score by making a task slower and more expensive — more retrieval, more reasoning, a bigger model, three passes instead of one. Whether that is worth it depends on numbers that have nothing to do with accuracy.
Our standard evaluation reports four columns per candidate configuration: accuracy split by error type, p95 latency, cost per task, and escalation rate. A configuration that is two points more accurate, 40% more expensive and twice as slow is a bad trade for order triage and an obvious win for a compliance check that runs nightly. The report makes that a business decision instead of an engineering preference.
A task is ready for autonomy when: silent-wrong sits under the agreed bar on a random sample of at least 500 items; confidence is calibrated within a few points across buckets; p95 latency fits the workflow; cost per task beats the manual baseline with margin; and the escalation path has been exercised at least once for real, by the actual person who will receive them.
That last one gets skipped constantly. An escalation route nobody has ever walked through is a diagram, not a control.
Recognise the process?
Bring one process that is eating your team's week. We will tell you on the call whether it is a good candidate — and we say no to roughly a third of what we are asked about.
