Measure chatbot usefulness beyond chat volume

Start with the reporting period#

Open the widget overview and check the period shown for each metric. Started chats, users, active conversations, and messages describe different behaviours. A total saved for the widget should not be compared directly with a daily chart without checking the scope.

Read a representative sample#

Review short conversations, longer conversations, unanswered questions, and successful product journeys. Look for whether the customer received a usable answer, reached a relevant page, or had to repeat the same question. More messages can reflect either engagement or difficulty.

Group repeated issues#

Use a small set of categories to choose improvements.

• Missing information: add or refresh the relevant source.
• Wrong model or variant: clarify product identity and attributes.
• Unhelpful answer: provide a verified correction and a better next step.
• Installation or usability: test placement, mobile layout, and consent behaviour.
• Follow-up needed: confirm the contact-capture and human response process.

Evaluate a change#

Compare similar periods and account for traffic or campaign changes. Repeat your answer tests and check new conversations for the original issue. Do not describe chat volume or a product click as a confirmed sale unless you have the appropriate purchase evidence.