It’s not wrong to think of developing conversational applications as the new frontier in interacting with your consumers and customers.

Why? Because it is the new frontier, just like smartphone apps were ten or so years ago.  

Know what's not new? 

The need to measure the success of the application you produce. Websites, smartphone apps, emails, social media channels, and now voice: their metrics all need to be analyzed to determine their success or failure. Yes, conversational AI is new, but digital analytics is not.  

Fortunately, you can follow established best practices— most of which were developed during the creation and implementation of other technologies that were once new— to ensure conversational applications are meeting your organization’s goals.  

The Four Rules 

Many organizations have high hopes for dashboards and the myriad data points they offer. But can you be sure your conversational analytics meet the mark? You can start by following these four rules. 

Rule #1: Start with your goals 

What do you plan to achieve with your conversational application? What’s its purpose? What customer need will it fulfill? Are you trying to increase customer satisfaction? Increase revenues? Decrease call center volume?   

Taking some time to determine what you hope the conversational application will accomplish can help prevent one common pitfall regarding analytics—shooting an arrow and then drawing a bull’s eye around it.  

To follow this rule: Be clear about the project’s objectives and what outcomes you are looking to achieve. 

Rule #2: Determine how you measure success 

Start by understanding what the key performance indicators (KPI) for the conversational application are. What do you need to measure to determine if you’re meeting your goals?  

Are you developing a consumer services bot that is meant to be sticky and engaging? Then, the KPIs you care about might be engagement and retention. Alternatively, maybe you want to reduce the time a person engages with the chatbot to get an answer. Or, perhaps you would like to minimize repeat transactions with the chatbot.

Once you've determined your goals (rule 1), understand what the KPIs are to determine if your conversational assistant is meeting those goals.  

To follow this rule: Identify the key questions regarding the conversational assistant's performance and then determine what data are needed to answer those questions. 

Rule 3: Measure engagement  

Now that you’ve determined what your KPIs are and what data you need to capture, be certain that you can collect this data. Often, organizations identify KPIs only to learn that key information isn’t being captured or collected.  

To follow this rule: Determine if the right data can be collected. Then collect it.  
 

Rule 4: Review, adapt, improve 

Businesses change, requirements change, and high-level priorities evolve. That's why it is essential to periodically review the data collected by your conversational application. Do they still measure the goals the business hoped to achieve with this conversational application? Do the KPIs reflect the changes in your business environment and strategic goals?  

To follow this rule: Adjust what data are being collected to determine if the conversational application is a tactic that continues to help the business meet its new goals. If not, consider collecting different data to evaluate its usefulness, or even consider revising the application.  
 

Orbita Insights 

Now that you know the rules, how can Orbita help you follow them? 

Orbita Insights is an analytics tool that provides organizations with knowledge of the metrics that matter for their conversational applications. The tool allows organizations to define key performance indicators, measure the progress towards defined goals, and facilitate strategies towards optimizing and improving conversational experience metrics. 

It is also flexible enough to adapt to changes in success metrics as your organization's strategic goals change.  

Key capabilities 

Orbita Analytics is a feature-rich tool. Like all good analytics tools, it provides a library of charts, graphs, and reports that users can choose from as they review the data collected. 

In addition, analysts can build custom reports using a reusable library of visualization components. For example, they can assess which intents are being used most often.  

In this example, you can see that the Help is the top request. Knowing that data, the business has further insight into how the conversational application is being used. From this, decisions can be made on how to improve the conversational application. For example, do things need to be explained better? Is the Help intent assisting people in the conversational experience?  

With Orbita Analytics, data analysts can start probing the data by asking a question, such as “How many questions were not understood?”  

The insights gained from these results can help analysts determine if the conversational application is handling user requests properly or users are engaging with the conversational application in unexpected ways.