All leading IT companies have developed cloud-based platforms that allow building a chatbot in few steps and most times without knowledge about programming languages. These services are based on Natural Language Understanding (NLU) engines which deal with identifying information such as entities and intents from the sentences provided as input. In order to integrate a chatbot on an e-learning platform, we want to study the performance in intent recognition task of major NLU platforms available on the market through a deep and severe comparison, using an Italian dataset which is provided by the owner of the e-learning platform. We focused on the intent recognition task because we believe that it is the core part of an efficient chatbot, which is able to operate in a complex context with thousands of users who have different language skills. We carried out different experiments and collected performance information about F-score, error rate, response time and robustness of all selected NLU platforms.

A Comparison of Natural Language Understanding Services to build a chatbot in Italian

Matteo Zubani
;
Ivan Serina;Alfonso Emilio Gerevini
2020-01-01

Abstract

All leading IT companies have developed cloud-based platforms that allow building a chatbot in few steps and most times without knowledge about programming languages. These services are based on Natural Language Understanding (NLU) engines which deal with identifying information such as entities and intents from the sentences provided as input. In order to integrate a chatbot on an e-learning platform, we want to study the performance in intent recognition task of major NLU platforms available on the market through a deep and severe comparison, using an Italian dataset which is provided by the owner of the e-learning platform. We focused on the intent recognition task because we believe that it is the core part of an efficient chatbot, which is able to operate in a complex context with thousands of users who have different language skills. We carried out different experiments and collected performance information about F-score, error rate, response time and robustness of all selected NLU platforms.
File in questo prodotto:
File Dimensione Formato  
paper36.pdf

accesso aperto

Descrizione: Articolo pubblicato
Tipologia: Documento in Post-print
Licenza: PUBBLICO - Creative Commons 4.0
Dimensione 451.49 kB
Formato Adobe PDF
451.49 kB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11379/536135
 Attenzione

Attenzione! I dati visualizzati non sono stati sottoposti a validazione da parte dell'ateneo

Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 1
  • ???jsp.display-item.citation.isi??? ND
social impact