Improving NLU model

It's a common situation that you can not foresee all possible phrases, intents, and entities in your conversation script. Thus, it would be best if you improve your NLU model and script all the time.

You need to validate your model on errors and fix them regularly because it can make mistakes. Do it after a batch of new conversations. When you create an app, let users talk to it, then analyze conversations, check if there were any mistakes, fix them and let users talk to your app again. Repeat this process until you are happy with how the app performs.

To improve the model, extend training examples with new correct data. You should fill data.json file with additional data for intents and entities.

To make validation process easier you should use Profiler in DashaStudio or within a Standalone app.

There are actions you can do to improve NLU model:

If the model classified unseen phrases correctly, you should add them to includes training examples of corresponding intent. It will extend the domain of this intent and increase the classification precision of these phrases and other as yet unseen phrases. Don't forget to markup entities in this phrase.

False-positive classification error means that intent or entity was recognized but really shouldn't have. This error affect conversational script, and Dasha app start behave wrong. Application go to wrong path and may answer not that was expected.

If the phrases don't have recognized meaning (intent), you should add it to excludes training examples of corresponding intent. Then the algorithm will know that and make these phrases less similar to phrases from the includes list and won't recognize this intent.

If entity was recognized from words, that actually are not entity, you need to add this value to excludes of corresponding entity. Then, algorithm will know that this value is not possible for the entity.

False-negative classification error means that intent or entity was NOT recognized but really should have. Usually, the reason is that there are not enough similar phrases for this intent.

If intent was not recognized from the phrase, you should add it to includes training examples of corresponding intent. It will extend the domain of this intent and increase the classification precision of these phrases and other unseen phrases.

If entity was not recognized from the phrase, you should add this phrase to dataset and markup all entities correctly. If entity value was not in list of values, add it too.

Take into account that the user can have some other intention and entities that you didn't use previously, you didn’t expect them to say or ask certain things. When you validate conversations, you can notice that there are many similar phrases of new meaning. Then you need to create new intent or entity. Don't forget to use them in your conversational script.