You are considering building a chatbot. If you decide to build an artificial intelligence (AI) powered chatbot you will undoubtingly question yourself the following: which AI/NLU engine do I pick? The answer to this question is not as straightforward as you might think. First of all, there is no such thing as the best engine. The performance of the AI used under the hood of your chatbot depends on the use case of the chatbot. This by itself is not the issue. The real issue here is that you will never know the performance of your chatbot until you have built it and put it into production. As every organization is different, and as such each organization has a specific use case, every chatbot is different too. An airplane manufacturer uses very different words than a fastfood chain. Moreover, a fastfood chain that sells chicken is very different from one that sells donuts. Hence, the performance of your chosen ‘NLU – use case’ combination depends on the words used within the organization. This makes predicting the performance of your ‘NLU – use case’ combination very difficult upfront.
Gambling with NLU engines and resources
And why is this an issue? Firstly, there is hardly anything as annoying as a malfunctioning chatbot. It literally scares away customers. Secondly, once you have found out that your chatbot is underperforming, you need to reinvest all lots of time and resources again to simply try again. After all, you do not know upfront if the second NLU technology will be better. Thirdly, and more practically, if you decide to do the development of your chatbot in-house, you probably have hired experts in one of the NLU engines, but if this technology does not bring the desired results, you cannot simply let them work on another NLU engine. For example, Google Dialogflow is configured very differently than Microsoft LUIS or Rasa. Fourth, the development of NLU engines is an ongoing process. This means that your ‘NLU – use case’ combination might be unmatched today, but tomorrow a competing NLU might suit your use case better. You do not want to get outdated right? Fifth and last, you might get it right at once. Great! You have a high-performance chatbot in one language. The entire process starts again for any additional language.
Create the best model with knowledge instead of luck
As is demonstrated by the above, the creation of a long-term reliable and high performance chatbot is far from easy. However, things do not need to be so difficult if you use the Blits platform. All you need is a dataset with sentences that your audience is likely to reply to the chatbot. Simply enter around 70% of these sentences to the platform and train your desired NLU engines. Afterwards, the ‘NLU Performance Test’ feature can be used to test the performance of each model on both the trained sentences and the roughly 30% of untrained sentences. Simply pick the NLU model that produces the highest scores to get the best results.
Monitoring the theoretical model in a practical world
Now that you have created and selected the best NLU model, it is time to put your model to the test. Although you have carefully thought of you all the possible user inputs and have trained your model using them, you will find your users entering unexpected inputs. The ability to deal with these unexpected demands differs per NLU model. For this it relies on information outside of the trained model, such as the prebuilt entities. Monitoring how well each model responds outside of the trained model can also be done using the ‘Intent Detection’ feature. This feature measures the recognition of user input across one or more NLU engines simultaneously. This way, you can real-time observe the performance of your chosen NLU engine and benchmark it’s performance against other NLU engines through running experiments. Experiments can be done on a regular basis to ensure your chosen NLU engine is still the best available NLU engine for your ‘NLU – use case’ combination.
Through these unique Blits features, you make sure your chatbot audience, whether those are your internal employees, suppliers for your organization or customers of your product, receives a state-of-the-art chatbot experience, while you experience the benefits of digitizing communication processes Paul Coerkamp wrote about in his blog.
*: In this blog, the term Natural Language Understanding (NLU) was used instead of the more commonly used Natural Language Processing (NLP). Generally, NLU is narrower in purpose because it focusses primarily on machine reading comprehension: getting the computer to comprehend what a body of text really means. As such, it is considered a subtopic of NLP.
