Your virtual assistant monitors your environment’s metrics, from conversations and knowledge bases to external interactions and license management. This page describes the analytics reports and their purposes.

Prerequisites

To use this functionality, a user must have the Koji admin or Koji analyst role. To manage roles, go to Configuration > Tenants > Tenant role setup:

Open

Applications

Applications are the Kbot components that allow you to associate specific parts of Kbot with authentication methods. This functionality allows you to limit the access to particular Kbot parts, or create multiple chats for different purposes.

Applications usage history

See the visualization of how often users interact with each application within a certain period of time.

Conversation training

Conversation training is a tool for analyzing real-life conversations and improving the bot’s ability to provide useful replies.

Answer ratio

View the number of cases when the bot is able to reply to users. The less the “bot has no answer” count is, the better.

Good answer ratio

This report is similar to the Answer ratio report, but it allows you to view the number of cases when the bot has the good answer. The less the “bot does not have a good answer” count is, the better.

Most returned articles

View the articles that are returned the most in the trainer reviews of the chat training user statements.

Conversation review overview

Study the general overview on the conversation training. See the global metrics, such as reviews and the ratio of answers (both good and generic). You can also study the historical details of each metric presented as a percentage ratio or a count.

Review status

See the statistics on the reviewed and not reviewed conversations to evaluate how effective the reviewing process has been.

Typology of best answer responses

View the bot’s responses marked as “Best“ by the trainer in Chatbot > Conversation training. This report allows you to see the typology of replies suggested by the bot that are considered useful by an expert.

Typology of bot responses

View the bot responses that are displayed in Chatbot > Conversation training. You can see the typology of the replies suggested by the bot and figure out which of them are considered useful by an expert. Also this statistics allows you to identify whether an expert missing something important.

Conversations

Conversation activity over working hours

View the number of user statements processed each hour of the day for the selected period. The report provides the visualization, so you can quickly figure out the busiest hours.

Chat sessions by channel

View the details on chat sessions that took place in various channels.

Conversation for an ID

View details on a particular conversation by specifying a conversation ID or user statement ID.

Conversations for one intent

View the conversations with a particular intent used.

Conversations journeys

Conversations journeys help to understand how users are exploiting the chat: which intents, topics, and actions are used and how often they are used.

This Conversations journeys report displays the most common conversation paths.

Number of conversations

Check the number of conversations within a certain period of time. Besides the number of conversations, you can check the number of discussed topics and exchanged messages.

Conversations quality history

Study the history of user statements processing: see the total number of user statements and the corresponding percentage of statuses. The results are grouped by week and user statement statuses.

User statements routing

This report visualizes the way Kbot manages conversations. View the graph that represents conversations and the bot’s decisions.

The categorization method that is applied on each level is defined in a JSON file, so it is possible to configure various options for your environment to offer a different kind of data for analysis.

User statements status

This report is designed to help you understand how user statements with a particular status (completed/abandoned/failed) are handled by the bot. This visualization helps you to see the story behind the status classification.

Chat session volume history

View the details on chat sessions that took place within a certain period of time. You can check the number of sessions, discussed topics, and exchanged messages.

External interactions

As your virtual agent can interact with external systems, it collects the corresponding statistics. Thus, you can understand which transactions are the most popular, and monitor when they took place.

External system transactions over time

View the external system transactions performed within a certain period of time. The report shows you the number of transactions and their types.

Intents

Intent in Kbot is the primary logical unit of a conversation. The main idea of a conversation processing is extracting an intent and delivering a corresponding response to a user. 

Most common intents visualization

View the word cloud graph made of the most common intents that have been extracted from conversations.

Intents usage by topic

View the number of times the intents for each topic have been triggered.

Intent usage count

View the number of times intents have been triggered within a certain period of time.

Knowledge bases and search contexts

A knowledge base is a set of articles from a specific source. Kbot provides users with these articles to answer their questions and help to solve various problems.

A search context defines a search strategy in the knowledge base. Search contexts allow to fine-tune knowledge bases, as you can specify the search parameters, such as the language, article components.

The reports in the Knowledge bases section help you identify the most popular articles, view both knowledge bases and search contexts usage, and so on.

Article click details

The report presents the list of articles that were opened by users. You can view user statements and associated articles' details: titles, knowledge bases, IDs, and URLs.

Most viewed articles

View the most popular articles and study their details, such as article ID, title, corresponding knowledge base, and language.

Search contexts usage

View the number of times each search context has been used to find articles.

Knowledge base usage

View the number of searches each knowledge base has performed.

User statements

View user statements that Kbot responded to with knowledge base articles.

Knowledge training

Users can mark articles that are returned by Kbot as “useful” or “not useful”. This feedback builds a knowledge base called the Recommender system. It offers useful articles to users when appropriate, speeding up the search.

Training activities per day

View the number of training activities per day and check the ratio of "useful" marks in the chat and knowledge training tools.

Size of training sets per knowledge base

See the number of articles, questions, and the corresponding average number of questions per article.

Live agents

Activity by operator

View the activities carried out by the operator within a certain period of time.

Monthly activity

View the summary of operators' activities per month.

Request for operator history

View how often users request for an operator in conversations. Check the amount of failed requests because of no operators are available.

Operator sessions

View the statistics on operator sessions within a certain period of time.

Surveys

It is important to know whether users are satisfied with Kbot. The Survey feature allows you to run client satisfaction surveys to identify how users evaluate the virtual assistant’s performance.

Historical trend in surveys

View the summary of survey results over time.

User comments on surveys

View the details on user comments.

Tickets

Tickets created per day

View the number of tickets that have been created daily.

Submitted tickets details

View the details on created tickets: when a ticket was created, the user who created it, this ticket’s type, category, and ID.

Users

Discover new users

View information about new users and check their activity. You can send emails to all the users of the selected tenant.

New and returning users

View the number of new (first connection with the bot) and returning users.

Real and visitors

View the list of active users and visitors who exist in the database.

Score of users

This report is leveraging the concept of gamification. Now users have scores that are based on the number of sessions, discussed topics, and exchanged messages. The more a user interacts with Kbot, the higher their score is.