Note:Touse these metrics and attributesin custom reporting, you need a Zendesk Explore Professional or Enterprise plan. Accounts on theExplore Lite planhave access to a subset of these metrics and attributes throughprebuilt dashboardsonly.
Use this article to discover the metrics and attributes you can use to build Explore reports based on your usage of Zendesk Answer Bot. These datasets are also used for the Answer Bot prebuilt dashboards (seeOverview of the Answer Bot dashboard).
For more information about how to create reports with Explore, seeCreating reports.
This article contains the following topics:
Article Recommendations dataset
This section contains the following topics:
Article Recommendations schema
Use this diagram to help you understand the elements of the Article Recommendations dataset and their relationships.
Article Recommendations metrics
This section lists and defines all metrics available in the Article Recommendations dataset.
| Metric | Definition | Explore formula |
|---|---|---|
| Attempts | All instances in tickets and chats where Article Recommendations was available to recommend articles, regardless of whether the bot identified relevant articles to suggest. | [Answer ID] |
| Answers | Instances in tickets and chats where the bot identified articles relevant to a customer’s query and suggested them to the customer. | IF ([Answer status - Unsorted]="Offered" OR [Answer status - Unsorted]="Clicked" OR [Answer status - Unsorted]="Resolved") THEN [Answer ID] ENDIF |
| Unsuccessful attempts | Instances in tickets and chats where the bot could not identify any articles relevant to a customer’s query and did not make a suggestion to the customer. | IF ([Answer status - Unsorted]="Unoffered" ) THEN [Answer ID] ENDIF |
| Clicks | Counts successful Answer Bot attempts where at least one generated suggestion was clicked. | IF ([Answer status - Unsorted]="Clicked" OR [Answer status - Unsorted]="Resolved") THEN [Answer ID] ENDIF |
| Resolutions | Count of Answer Bot answers that resolved the end user’s request. | IF ([Answer status - Unsorted]="Resolved") THEN [Answer ID] ENDIF |
| Indirect resolutions | The number of Answer Bot answers that indirectly resolved the end user’s request. In some situations, articles suggested by Answer Bot don’t resolve the customer’s enquiry directly but instead point them in the right direction in the help center to find the answer to their question. These resolutions are known asindirect resolutions. Indirect resolutions are reflected inResolutions,Indirect resolutionsandAnswer statusfields. |
D_COUNT(Resolutions)-COUNT(Resolution articles) |
| % Suggestion rate | The percentage of enquiries where Answer Bot offered suggestions from the total number of enquiries it attempted to answer. | D_COUNT(Offered answers)/D_COUNT(Attempted answers) |
| % Click-through rate | The percentage of answers clicked by end users from the total answers offered by Answer Bot. | D_COUNT(Clicked answers)/D_COUNT(Offered answers) |
| % Resolution rate | The percentage of enquiries resolved by Answer Bot from the total enquiries where it offered suggestions. | COUNT(Resolutions)/D_COUNT(Offered answers) |
| Suggested articles | Count of the suggestions made by Answer Bot. | [Suggestion ID] |
| Clicked articles | The number of Answer Bot suggestions that have been clicked by a user. |
IF ([Suggestion clicked]="true" ) THEN [Suggestion ID] ENDIF |
| Resolution articles | Count of solved tickets solved through a suggestion from Answer Bot. | IF ([Suggestion resolved]="true" ) THEN [Suggestion ID] ENDIF |
| Rejected articles | The number of Answer Bot suggestions that have been marked as unhelpful by end users. | IF ([Suggested article rejected]=TRUE) THEN [Suggestion ID] ENDIF |
| % Article click-through rate | The percentage of Answer Bot suggestions that have been clicked by an end-user. | COUNT(Clicked suggestions)/COUNT(Suggestions) |
| % Rejection rate | The percentage of Answer Bot suggestions that have been marked as unhelpful by end users. | COUNT(Rejected articles)/COUNT(Suggested articles) |
| Answered tickets | Count of tickets where an Answer Bot response was generated, a suggestion was clicked, and this resulted in the ticket being solved. | IF ([Answer status - Unsorted]="Offered" OR [Answer status - Unsorted]="Clicked" OR [Answer status - Unsorted]="Resolved" ) THEN [Answer ticket ID] ENDIF |
| Unanswered tickets | Count of tickets where an Answer Bot response was not given. | IF ([Answer status - Unsorted]="Unoffered" ) THEN [Answer ticket ID] ENDIF |
| Answer clicked tickets | Count of tickets where an Answer Bot response was generated, which resulted in the ticket being resolved. | IF ([Answer Status - Unsorted]="Clicked" OR [Answer status - Unsorted]="Resolved" ) THEN [Answer ticket ID] ENDIF |
| Solved tickets | Count of tickets with a status ofSolved. |
IF ([Answer status - Unsorted]="Resolved") THEN [Answer ticket ID] ENDIF |
| Non-reopened solved tickets | Count of tickets with a status ofSolved, that have not been re-opened. | IF ([Answer status - Unsorted]="Resolved" AND VALUE(Reopens)<1) THEN [Answer ticket ID] ENDIF |
| Reopened solved tickets | Count of tickets that were set to a status ofSolvedwith Answer Bot, and were subsequently re-opened. | IF ([Answer status - Unsorted]="Resolved" AND VALUE(Reopens)>0) THEN [Answer ticket ID] ENDIF |
| Agent unassisted solved tickets | Count of tickets with no Assignee set, and with a status ofSolvedthrough an Answer Bot suggestion. | IF ([Answer status - Unsorted]="Resolved" AND VALUE(Agent replies)<1) THEN [Answer ticket ID] ENDIF |
| Agent assisted solved tickets | Count of tickets that are assigned to an agent, where the agent has replied, and the ticket status isResolvedusing an Answer Bot suggestion. | IF ([Answer status - Unsorted]="Resolved" AND VALUE(Agent replies)>0) THEN [Answer ticket ID] ENDIF |
| % Ticket usage rate | The rate of tickets that were assisted by Answer Bot suggestions in comparison to total number of tickets. | DCOUNT_VALUES([Answer ticket ID])/DCOUNT_VALUES([Ticket ID]) |
| % Ticket resolution rate | The percentage of tickets resolved from Answer Bot suggestions compared to the total solved ticket volume where an Answer Bot response was given. | D_COUNT(Solved tickets)/D_COUNT(Answered tickets) |
| % Ticket non-reopened resolution rate | The rate of ticket resolutions provided by Answer Bot suggestions in comparison to total tickets where an Answer Bot answer was generated. | D_COUNT(Non-reopened solved tickets)/D_COUNT(Answered tickets) |
| % Ticket unassisted resolution rate | The rate of ticket resolutions that have not been re-opened provided by Answer Bot suggestions in comparison to total tickets where an Answer Bot answer was generated. | D_COUNT(Agent unassisted solved tickets)/D_COUNT(Answered tickets) |
| Click time (min) | The duration in minutes between a suggestion being generated in Answer Bot, and this suggestion being clicked. | (Click time (min)) |
| Resolution time (min) | The duration in minutes between a suggestion being generated in Answer Bot, and this suggestion solving the ticket. | (Resolution time (min)) |
| Click time (hrs) | The duration in hours between a suggestion being generated in Answer Bot, and this suggestion being clicked. | VALUE(Click time (min))/60 |
| Resolution time (hrs) | The duration in hours between a suggestion being generated in Answer Bot, and this suggestion solving the ticket. | VALUE(Resolution time (min))/60 |
| Agent replies | Counts the total number of agent replies to a ticket. | (Agent replies) |
| Reopens | The number of times a ticket status changed fromSolvedtoOpen. | (Reopens) |
Article Recommendations attributes
本节avai列表和定义所有属性lable in the Article Recommendations dataset. You can use this diagram to help you understand the meaning of some of the attributes in the list:
| Attribute | Definition |
|---|---|
| Answer status | The status of a suggestion provided by Answer Bot. The status is set depending on the end-user's response. Values includeUnoffered,Offered,ClickedorResolved. |
| Answer channel | 回答Bot interac通道ted with the end user or agent. Values includeEmail,Web Widget,Web form,API,SDK,Slack, andAgent(which is when an agent uses the Knowledge Capture app or the knowledge section of the context panel from within a ticket). |
| Answer brand | The brand of ticket corresponding to the brand of the knowledge base from which Answer Bot generated suggestions for an answer. |
| Answer enquiry | The end user’s question for which Answer Bot looked for suggestions. The attribute values are limited to the first 255 characters. |
| Answer ID | The ID of the notification with suggestions sent by Answer Bot. |
| Answer resolution article ID | The ID of the knowledge base article which resolved the end user’s enquiry. |
| Answer ticket ID | The ID of the ticket from which the Answer Bot response was triggered. |
| Suggested article clicked | The event of the suggestion generated by Answer Bot being clicked by an end-user. Values areTrueandFalse. |
| Suggested article resolved | The event of the suggested article being clicked from the answer generated by Answer Bot which led to the ticket being resolved. Values areTrueandFalse. |
| Suggested article rejected | The article suggestion generated by Answer Bot being marked as unhelpful by an end-user. Values areTrueandFalse. |
| Suggestion ID | The ID of the knowledge base article suggestion sent by Answer Bot. |
| Suggested article language | The language of the knowledge base article suggestion sent by Answer Bot. |
| Suggested article locale | The locale of the knowledge base article suggestion sent by Answer Bot. Values includeEN-US,EN-GB,DE,FR,RU. |
| 文章的ID | The ID of the knowledge base article. |
| 文章的IDand locale | The ID and locale of the knowledge base article. |
| Article translation title | 在speci知识库文章的标题亚博官方appfic language. |
| Article translation URL | The URL of the knowledge base article. |
| Article author | The name of the user who originally created a knowledge base article. |
| Ticket ID | The ID number of the ticket. |
| Ticket status | The current status of the ticket. |
| Ticket group | The name of the group where the ticket was assigned. |
| Ticket assignee | The name of the user to who the ticket is assigned. |
| Ticket brand | The brand of the ticket. |
| Ticket channel | The channel a ticket was created from. For more information about the ticket channels Explore collects, seeUnderstanding ticket channels in Explore. |
| Ticket form | Ticket form used on the ticket. |
| Ticket organization | The name of the organization associated with the ticket. |
| Ticket priority | The ticket’s priority. |
| Ticket problem ID | The ID of the associated problem ticket. |
| Ticket requester | The name of the user that requested the ticket. |
| Ticket satisfaction rating | The satisfaction rating of the ticket,GoodorBad. |
| Ticket subject | The subject of ticket. |
| Ticket tags | The tags associated with a ticket. |
| Ticket type | The ticket type. |
| Sharing agreement inbound | Affiliated instances of Zendesk Support and companies who share tickets with the current instance of Zendesk Support. |
| Sharing agreement outbound | Affiliated Zendesk accounts and companies tickets are shared with. |
| User name | The name of the user who was assisted by Answer Bot. |
| User role | The role of the user who was assisted by Answer Bot. |
| User ID | The ID of the user who was assisted by Answer Bot. |
| User email | The email address of the user who was assisted by Answer Bot. |
| User locale | The locale of the user who was assisted by Answer Bot. |
| User status | The Zendesk status of the user who was assisted by Answer Bot. Values areActive,SuspendedorDeleted. |
| User tags | A list of tags associated with the user who was assisted by Answer Bot. |
| User time zone | The time zone of the user who was assisted by Answer Bot. |
| User organization name | The organization name of the user. |
| User organization ID | The organization ID of the user. |
| User organization domains | The domain name of the organization of the user, for example, zendesk.com. |
| User organization status | The system status of the organization of the user, either active or deleted. |
| User organization tags | The tags associated with the organization of the user. |
| Time – Bot answer created | Includes several attributes that refer to the time when an answer was generated by Answer Bot containing relevant knowledge base articles. |
| Time - Bot answer last event | Includes several attributes that refer to the last answer notification from Answer Bot. |
| Time - Bot suggestion last event | Includes several attributes that refer to the last time Answer Bot triggered a suggestion. |
| Time – Bot suggestion clicked | Includes several attributes that refer the time a suggested article was clicked from an answer generated by Answer Bot. |
| Time – Bot suggestion resolved | Includes several attributes that refer to the time a suggested article was clicked from an answer generated by Answer Bot, which led to ticket being resolved. |
| Time – Ticket created | Includes several attributes that return the time and date when tickets were created. |
| Time – Ticket solved | Includes several attributes that return the time and date when tickets were solved. |
| Time – Ticket last updated | Includes several attributes that return the time and date when tickets were last updated. |
| Time – Article created | Includes several attributes that return the time when an article was created. |
| Time – Article last updated | Includes several attributes that return the time when an article was last updated. |
Flow Builder dataset
This section contains the following topics:
Flow Builder metrics
This section lists and defines all metrics available in the Flow Builder dataset.
| Metric | Definition | Explore formula |
|---|---|---|
| Total users | The number of unique users who received a message from Answer Bot. A user does not have to engage with the bot (sending a message or clicking a quick reply) to be included in this count. | [Sunshine Conversations app user ID] |
| Engaged with bot | The number of unique users who have sent a message to Answer Bot or responded to a prompt. | IF ([Activity event type] = "user_input_received") THEN [Sunshine Conversations app user ID] ELSE NULL ENDIF |
| Transferred to agent | The number of unique users who have successfully completed the transfer step from Answer Bot to an agent. A Support ticket is created for each successful transfer. | IF ([Activity event type] = "agent_transfer_completed") THEN [Sunshine Conversations app user ID] ELSE NULL ENDIF |
| % total users who engaged with bot | The percentage of total users who have sent a message to Answer Bot or responded to a prompt. | D_COUNT(Engaged with bot) / D_COUNT(Total users) |
| % engaged users transferred to agent | The percentage of engaged users who have successfully completed the transfer step from Answer Bot to an agent. | D_COUNT(Transferred to agent) / D_COUNT(Engaged with bot) |
| User input count | The total number of user inputs (messages of any type) received by the bot. | [Content text] |
| Total users in previous 30 days | The total number of users in the period from 62 days to 32 days before today. | |
| Total users in last 30 days | The total number of users in the period from 31 days ago to yesterday. | |
| Engaged users in previous 30 days | Users who engaged with the bot in the period from 62 days to 32 days before today. | |
| Engaged users in last 30 days | Users who engaged with the bot in the period from 31 days ago to yesterday. | |
| Transferred users in previous 30 days | Users who were transferred from the bot to an agent in the period from 62 days to 32 days before today. | |
| Transferred users in last 30 days | Users who were transferred from the bot to an agent in the period from 31 days ago to yesterday. | |
| Containment rate | The proportion of engaged users whose conversation wasnottransferred from the bot to an agent. The inverse of% transferred to agent. | |
Resolution feedback prompts |
The number of times the bot asked for feedback via an “Ask if question resolved” step. Note:This metric and the following metrics are counted based on each instance of the event and not aggregated per user. So if a user went through the same flow and replied to a feedback prompt more than once, this metric counts each instance of a prompt being sent or a feedback reply interaction. |
|
| Resolution feedback replies | The number of feedback prompt replies received by the bot. | |
| % resolution feedback replies | The proportion of times the bot received a reply after sending a feedback prompt. This is the count of replies of any type divided by the count of prompts sent. | |
| Resolved feedback replies | The number ofresolvedfeedback prompt replies received by the bot (for example, “Yes, problem solved”). | |
| % resolved | The proportion of total replies that wereresolved回答。This is the count of resolved replies divided by the count of total replies. | |
| Unresolved feedback replies | The number ofunresolvedfeedback prompt replies received by the bot (for example, “No, I still need help”). | |
| % unresolved | The proportion of total replies that wereunresolved回答。这是未解决的回复d的计数ivided by the count of total replies. |
Flow Builder attributes
本节avai列表和定义所有属性lable in the Flow Builder dataset.
| Attribute | Definition |
|---|---|
| Brand ID | The ID of the brand associated with the bot the user engaged with. |
| Brand name | The name of the brand associated with the Flow Builder flow the user engaged with. |
| Sunshine Conversations app user ID | The unique identifier of the user participating in a messaging conversation. |
| Activity event type | The event type recorded as a result of certain bot conversation interactions. Values include: Conversation started (journey_started), Engaged with bot (user_input_received), Transferred to agent (agent_transfer_completed), (resolution_prompt_sent), (subflow_switched), and (user_feedback_received). Events are recorded only for published answers. If answers are removed, unpublished, or renamed, historical data isn’t affected. |
| Channel | The Zendesk channel that the Answer Bot interaction took place on. In Sunshine Conversations terminology, a “channel” is referred to as “source.type”. |
| Time - Activity occurred | Includes several attributes that return the time and date when an activity (Conversation started, Engaged with bot, or Transferred to agent) happened. |
| Language | The standard name for a language (for example, Simplified Chinese). |
| Language code | The shorthand code for a language (for example, zh-CN). |
| User input type | The type of user input received by the bot. Possible values includeOption selected,Message sent,Form sent, andUnknown. |
| Content type | This is effectively the same as user input type. User input type is the human friendly name, while content type is the raw value used by the system. |
| Content text | The text of the message sent to the bot. This includes only messages sent as a result of selecting a quick reply option (theOption selectedvalue in theUser input typeattribute). Free text entered by end users is returned as blank values. |
| Answer ID | The ID of the answer. |
| Answer name | The name of the answer. This is referred to as “intent” when creating an answer in Flow Builder. |
| Answer type | The type of answer. Possible answer types are:
|
| Option is resolution feedback | Indicates whether the type of option selected was a resolution feedback reply, as opposed to a standard option. Possible values aretrueorfalse. An attribute of theuser_input_receivedevent. |
| Resolution feedback reply | Indicates how the user replied to the resolution feedback prompt (when the bot asks for feedback via an “Ask if question resolved” step). Possible values areResolvedorUnresolved. An attribute of theuser_feedback_receivedevent. |
10 Comments
Thank you Zendesk. I am excited to dig into these analytics and improve our customer experience.
Do you happen to know if we will increase the count number of 1000 steps on the FlowBuilder? I believe once we have this fully optimized to our liking and the liking of our customers, this won't be sufficient for our needs. Does a higher plan allow you have access to more steps within the flow?
HiDallin Harmonwe have some major Flow Builder enhancements planned in the 9 month roadmap that will increase the number of steps that can be configured beyond the current limit. Plan type has no bearing on the current limit.
As a temporary solution we have a feature flag available that will extend the step limit from 1,000 to 2,000. However, please note that exceeding 1,000 steps may impact the performance of the Flow Builder admin interface.
That is really good to know. Thank you. How do we contact someone from Zendesk to get that flag put on our account. We will be very careful and cautions in adding additional steps.
Right now, as per the articleLimitations in messaging functionality -- Volume limitations, the number of steps in a Flow Builder bot flow is only up to 1000 steps at the moment.
I'm not sure about the feature flag that was previously mentioned by my colleague, but I recommend you submit a product request or feedback on the Flow Builder enhancements in theGeneral Product Feedback topicin our community to engage with other users who have similar needs and discuss possible workarounds. Conversations with a high level of engagement ultimately get flagged for product managers to review when they go through roadmap planning.
We truly value customer feedback and your voice and votes in the forums help influence future Zendesk functionality. Stay safe!
HiDallin Harmon, i'll reach out to you directly via email about the feature flag.
Hey there, do you happen to know if this data is available via an API?
I'm afraid there are no public facing APIs available for Answer Bot metrics. Currently, the only way to pull up these metrics is through Explore.
我有一些问题需要得到数据的% users who are selecting the "transfer to agent" option in our flow builder. I tried using "count" with the keyword of "Or, chat with an agent" but it was double counting it. I also tried using count of "activity event" attribute but it was not giving me count of users who transferred to agent
Currently I am able to get the % of users who opts to "transfer to agent" if it was in the 1st level of my flow builder where I have only 2 options: self service or transfer to agent
However, for users who selects the self service flow, i have 2 to 4 additional layers for self service where they can be self-guided in our articles based on common categories. User will subsequently be prompted to end the self service flow or choose to transfer to agent (if they have not found the answer to their question).
I'd like to know the ratio of users who are opting to transfer to agent on every flow provided to them (excluding the 1st level flow).
Hi Alexan,
Thanks for your feedback. We are currently working on the next iteration of the Flow Builder dataset and dashboard which will include capturing performance metrics for every answer flow so you have more granular insights. I believe this will provide what you're looking for. We currently expect to release this enhancement in late Q4.
HeyDaniel Aron,
Is there a more specific release date for this yet? We are in Q4 and without this reporting Flow Builder isn't a tool we can fully roll out.
Thanks,
Jake
Pleasesign into leave a comment.