NLU: Natural language understanding

tip

NLU is a field of Natural Language Processing concerned with understanding the meaning of statements made in natural language.

NLU is one of the key functions which chatbots must have. It enables them to interpret requests and react to them in accordance with their clients’ expectations.

Chatbots apply NLU algorithms to solve two basic problems: detecting the communicative intention (intent) of their interlocutor and recognizing any mentioned named entities.

JAICP provides the following ways of implementing NLU in bots:

• CAILA NLU
• Patterns

NLU engines

CAILA NLU

CAILA (Conversational AI Linguistic Assistant) is the JAICP NLU core for developing chatbots and voice assistants that understand natural language.

Patterns

Patterns are formal rules that describe key words and expressions.

You can use patterns to assign a client reply to one of the existing system states that defines state-specific reactions. Patterns can be configured via base and extended elements.

tip

Patterns are also used in combination with CAILA NLU. You can use patterns to specify a record for an entity dictionary or a training phrase for an intent.

NLU languages

Supported languages

When you create a project, the mandatory NLU language parameter determines the language the bot will understand. For every supported language, JAICP automatically implements the following:

• a library for tokenization and morphological parsing;
• built-in algorithms for intent recognition;
• a set of standard entities.

caution

All languages other than English, Russian, and Chinese only support the STS classification algorithm.

Language	Note
English	Supports paraphrasing training phrases.
Chinese	Does not support: • fuzzy search and normalization of CAILA entities. • the ~ and $morph advanced pattern elements.
Danish
Dutch
French
German
Greek
Italian
Japanese	Does not support the recognition of time and numbers written out in full.
Kazakh	Does not support the recognition of time and numbers written out in full.
Lithuanian	Does not support the recognition of time and numbers written out in full.
Polish
Portuguese
Romanian
Russian	Supports spell checking and paraphrasing training phrases.
Spanish
Ukrainian	Supports spell checking.

Other languages

If your project requires support for a language not provided by JAICP, you can connect an external NLU service with support for any other language and use it instead.

You can develop such a service yourself or use a third-party one. The external NLU service must comply with the Model API specification.

tip

Learn more about configuring an external NLU service

Bot script

JAICP allows the combined use of CAILA NLU intents, patterns, and example groups together in a single script. State activation rules defined using these engines have different priorities.

tip

Learn more about the activation rule priority

The mechanism for selecting activation rules when using intents, patterns, and example groups can also be redefined manually, using either:

• the selectNLUResult handler;
• the $context.nBest field.