Chatbot Performance Measure

From GM-RKB
Jump to navigation Jump to search

A Chatbot Performance Measure is a application performance measure for a chatbot.



References

2024

  • (Inbenta, 2024) ⇒ https://www.inbenta.com/articles/10-key-metrics-to-evaluate-your-ai-chatbot-performance/
    • QUOTE:
      1. "Indeed, your customers won’t talk to a bot like they do to a human. In the same way, your employees won’t tell an HR team member the things they would say to a bot. So you have to accept that this new communication channel (if it didn’t exist before) will bring its share of surprises."
      2. "As obvious as it may seem, a regular monitoring will help you improve the effectiveness of the solution. However, these KPIs should not be the only metrics taken into consideration when evaluating the overall impact of the solution."
    • NOTE:
      • It focuses on the effectiveness of AI chatbots in meeting their intended objectives, including improving customer care, extending online support availability, and enhancing customer understanding.
      • It includes a variety of key metrics such as self-service rate, indicating the percentage of user sessions resolved without needing further contact action, and performance rate, measuring the accuracy of chatbot responses.
      • It also considers usage rate per login, which tracks the volume of active user sessions on the chatbot against the average number of sessions on the website, to assess user engagement and adoption.
      • The bounce rate for a chatbot, which accounts for sessions where the chatbot was opened but not interacted with, is another crucial metric for evaluating chatbot effectiveness in capturing user attention.
      • It evaluates user satisfaction through the average grade given when evaluating the chatbot’s answers, balanced against the evaluation rate, which is the percentage of sessions providing feedback on the chatbot's responses.
      • The average chat time and goal completion rate are key metrics in understanding user interest and the chatbot’s efficiency in guiding users to complete specific actions, like filling out a form or following through on CTAs.
      • The non-response rate, measuring the frequency of the chatbot failing to provide relevant content in response to user queries, is critical for identifying areas where the chatbot’s knowledge base or understanding capabilities need improvement.
      • It includes measuring the average number of interactions per session to evaluate the Customer Effort Score, helping to identify if the chatbot engages users in too many steps to meet their needs.
      • Monitoring these metrics regularly is essential for continuously improving the chatbot's effectiveness and ensuring it adds value according to the initial goals set for the chatbot project.