Chatbot Analytics: 8 Metrics That Actually Matter
Deflection rate looks great on a slide but hides unhappy customers. These eight chatbot metrics show whether your assistant really solves problems, plus a weekly review routine.

In this article
Most chatbot dashboards lead with one flattering number: how many conversations the bot "handled" without a human. It climbs every month, everyone nods, and customers who gave up halfway through a chat are quietly counted as wins.
The chatbot metrics that matter answer a simpler question: did the customer get what they came for, and did the business get value from the conversation? In practice that comes down to eight numbers: resolution rate, handoff rate, CSAT, first response time, fallback rate, lead conversion, answer accuracy and repeat contact rate.
Here's what each one tells you, how to calculate it, and a weekly routine that keeps your assistant improving.
Key takeaways
- Deflection counts conversations that never reached a human, not problems that got solved. Always pair it with resolution and repeat contact.
- Track a small set of chatbot KPIs consistently instead of every number your tool can export.
- A 30-minute weekly transcript review finds more improvements than any dashboard on its own.
Why deflection alone misleads
Deflection (sometimes called containment) is the share of conversations that ended without a human agent. It's easy to measure, which is why it's everywhere.
The problem is what it can't see. A customer who asks about a refund, gets a vague answer and closes the chat counts as deflected. So does someone who gives up on chat and emails you instead. A bot that never offers a human option will post an excellent deflection rate and a poor customer experience.
Treat deflection as a volume indicator, not a success metric.
The 8 chatbot metrics worth tracking
| Metric | What it tells you | Simple formula | Warning sign |
|---|---|---|---|
| Resolution rate | Problems actually solved | Resolved conversations / total | High deflection, low resolution |
| Handoff rate | How often people are needed | Escalated conversations / total | Very low (hidden exit) or rising fast |
| CSAT for AI chats | How customers felt | Positive ratings / all ratings | AI CSAT well below human CSAT |
| First response time | Speed of the first reply | Median time to first reply | Long waits after handoff |
| Fallback rate | Gaps in your knowledge | "No answer" replies / questions | Same gaps every week |
| Lead conversion | Revenue impact | Qualified leads / sales chats | Leads captured, never followed up |
| Answer accuracy | Trustworthiness | Correct answers / reviewed answers | Confident but wrong replies |
| Repeat contact rate | Whether answers stuck | Same-issue returns / resolved | Rising after a content change |
1. Resolution rate
The share of conversations where the question was answered or the task completed. Define "resolved" up front: the customer confirmed, rated positively, or didn't return on the same topic within a few days. Without a definition, this number drifts.
2. Handoff rate
How often the assistant passes a conversation to your team. There's no single right level: a clinic or B2B service should expect more handoffs than a store answering shipping questions. Watch the trend and read the reasons, not just the count. Our guide to chatbot-to-human handoff covers when escalation should happen.
3. CSAT for AI conversations
Ask a one-question rating at the end of AI-only conversations and compare it with human-handled ones. A large gap usually points to tone, missing information, or a bot that holds on too long. The CSAT guide covers wording and timing.
4. First response time
AI replies are near-instant, so the interesting number is what happens after handoff. If customers wait two hours after the bot says "Let me connect you," the experience is slow. Track first response time for the bot and the human queue separately.
5. Fallback ("no answer") rate
How often the assistant says it can't help or gives a generic reply. Group these by topic. A cluster of unanswered questions about international shipping is a content task for this week, not a mystery.
6. Lead conversion from chat
How many sales conversations produced a qualified lead, and how many of those became a booking, demo or order. If you collect contact details but nobody follows up quickly, the chatbot isn't the bottleneck.
7. Answer accuracy
Grade a sample of AI answers as correct, partly correct or wrong against your own sources. This protects your brand: a confidently wrong answer about a return policy costs more than no answer.
8. Repeat contact rate
Customers who come back about the same issue within a set window, such as seven days. Resolution rate shows obvious failures; repeat contacts catch the hidden ones.
How to measure chatbot performance in a weekly review
Dashboards tell you where to look. Transcripts tell you what to fix. Block 30 minutes a week and follow the same routine:
- Compare the trend for resolution, handoff, CSAT and fallback rate against last week.
- Read 15–20 transcripts, mixing low-rated chats, handoffs and a few random ones.
- Grade accuracy on those samples with the correct / partly correct / wrong scale.
- List the top three unanswered topics from fallback replies.
- Fix content, not just settings: add or update the knowledge source behind each gap.
- Review handoff reasons with whoever works the human queue. Was the timing right, and was there enough context?
- Log one change and check its effect next week.
After a month, your log shows which changes moved the numbers and which didn't.
Chatbot KPIs and targets: start from your own baseline
Resist copying a target from someone else's slide deck. Your question mix, industry and knowledge base decide what's realistic.
Measure for two to four weeks, then set goals relative to that baseline. For example, if you handle 1,200 conversations a month and 180 end in "I don't know," a sensible goal is to close the five biggest content gaps behind those 180, not to chase an arbitrary percentage.
Segment where you can: by channel (website chat, WhatsApp, Instagram), by topic and by language.
How Conviro supports chatbot analytics
Conviro's AI Assistants answer from your own knowledge base, and the built-in analytics cover the numbers above that matter most: CSAT, resolution rates and AI accuracy. When a conversation is handed off, it lands in the Live Inbox with the transcript, detected intent, tags and collected details, which makes reviewing handoff reasons quick.
If you run several brands, the multi-brand workspace keeps each assistant's knowledge base and analytics separate, so one brand's numbers don't hide another's problems.
Frequently Asked Questions
What is a good resolution rate for a chatbot?
There's no universal benchmark worth trusting, because resolution depends on your question mix and how complete your knowledge base is. A store answering shipping and returns questions can resolve far more than a clinic handling sensitive scheduling. Measure your own baseline for a few weeks, then improve it by closing the biggest content gaps.
What's the difference between containment and resolution?
Containment, or deflection, counts conversations that never reached a human. Resolution counts conversations where the customer's problem was actually solved. A chat can be contained but unresolved, for example when a frustrated customer simply leaves. Track both numbers and treat a widening gap between them as a warning sign.
How many chatbot conversations should I review each week?
For most small and mid-sized teams, 15 to 20 transcripts a week is enough to spot patterns without becoming a chore. Mix low-rated conversations, handoffs and a few random samples. After a big knowledge base change, review more for two weeks.
Which chatbot KPIs should I report to leadership?
Keep it to four: resolution rate, CSAT for AI conversations, lead conversion from chat and answer accuracy from your weekly sample. Together they show customer outcome, experience, revenue impact and risk. Add one line on the biggest change you made that month and its effect.
Next steps
Pick four of these eight metrics, record a baseline this week, and put a recurring 30-minute review in your calendar. If you want an assistant whose analytics and transcripts make that review easy, start free with Conviro.
Published by
Conviro
Sharing insights on AI-powered customer support, chatbot automation, and SaaS growth strategies.
Put the playbook into practice
Turn your support knowledge into faster answers
Explore Conviro’s AI support workspace, then start free when the workflow fits your team.
Related articles
More Posts
Customer Success
Knowledge Gap Analysis: Find the Questions Your Chatbot Can't Answer
7 min read
Read article
Customer Success
CSAT Explained: How to Measure and Improve Customer Satisfaction in Chat
6 min read
Read article
Customer Success
Proactive Customer Service: How to Reach Customers Before They Ask
7 min read
Read article