Resolution rate vs deflection rate: the AI support metric that actually matters
Deflection makes dashboards look great and customers miserable. Here's how to measure an AI support agent honestly, and the four numbers worth watching every week.
Ask most chatbot vendors how well their product works and you'll hear a deflection rate: the share of conversations that never reached a human. It sounds like success. Often it isn't. A customer who gives up and closes the chat counts as 'deflected' too.
Why deflection is the wrong goal
Deflection measures whether your team was avoided, not whether the customer was helped. A bot that loops endlessly, hides the 'talk to a person' option or answers a different question entirely can score very well on deflection. The cost shows up elsewhere: repeat contacts, refused deliveries, chargebacks and reviews.
If a metric improves when customers give up, it's measuring the wrong thing.
What a real resolution looks like
We define a resolution strictly: the AI agent closed the conversation without a human stepping in, and the customer didn't come back about the same issue within 72 hours. That second condition does most of the work. It filters out answers that sounded right but weren't, and customers who quietly gave up.
The four numbers worth watching
- Resolution rate: resolved conversations divided by all conversations the agent handled.
- Reopen rate: how often 'resolved' conversations come back within 72 hours. Rising reopens mean answers are wrong or incomplete.
- Handover rate, by reason: customer asked, low confidence, rule triggered, frustration. Each reason points to a different fix.
- Time to first reply: for customers, the biggest difference between AI and a queue.
Read handovers as a to-do list
A handover isn't a failure, but a pattern of handovers is information. Lots of low-confidence handovers on one topic usually means a missing or contradictory help page. Lots of rule-triggered handovers may mean an approval limit is set too low. Fix the cause and the resolution rate rises on its own, without pushing anyone away.
This is also why Onfre bills per resolution, not per conversation or per deflection. We only get paid when the customer actually got what they needed.