Skip to main content

2021 | OriginalPaper | Buchkapitel

Using BERT Model for Intent Classification in Human-Computer Dialogue Systems to Reduce Data Volume Requirement

verfasst von : Hao Liu, Huaming Peng

Erschienen in: Advances in Neuroergonomics and Cognitive Engineering

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

User-intent classification is a sub-task in natural language understanding of human-computer dialogue systems. To reduce the data volume requirement of deep learning for intent classification, this paper proposes a transfer learning method for Chinese user-intent classification task, which is based on the Bidirectional Encoder Representations from Transformers (BERT) pre-trained language model. First, a simulation experiment on 31 Chinese participants was implemented to collect first-handed Chinese human-computer conversation data. Then, the data was augmented through back-translation and randomly split into the training dataset, validation dataset and test dataset. Next, the BERT model was fine-tuned into a Chinese user-intent classifier. As a result, the predicting accuracy of the BERT classifier reaches 99.95%, 98.39% and 99.89% on the training dataset, validation dataset and test dataset. The result suggests that the application of BERT transfer learning has reduced the data volume requirement for Chinese intent classification task to a satiable level.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Chen, H., Liu, X., Yin, D., Tang, J.: A survey on dialogue systems: recent advances and new frontiers. SIGKDD Explor. Newsl. 19, 25–35 (2017)CrossRef Chen, H., Liu, X., Yin, D., Tang, J.: A survey on dialogue systems: recent advances and new frontiers. SIGKDD Explor. Newsl. 19, 25–35 (2017)CrossRef
2.
Zurück zum Zitat Schuurmans, J., Frasincar, F.: Intent classification for dialogue utterances. IEEE Intell. Syst. 35, 82–88 (2020)CrossRef Schuurmans, J., Frasincar, F.: Intent classification for dialogue utterances. IEEE Intell. Syst. 35, 82–88 (2020)CrossRef
3.
Zurück zum Zitat Shridhar, K., et al.: Subword semantic hashing for intent classification on small datasets. arXiv, pp. 1–6 (2018) Shridhar, K., et al.: Subword semantic hashing for intent classification on small datasets. arXiv, pp. 1–6 (2018)
4.
Zurück zum Zitat Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., Kuksa, P.: Natural language processing (Almost) from scratch. J. Mach. Learn. Res. 12, 2493–2537 (2011)MATH Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., Kuksa, P.: Natural language processing (Almost) from scratch. J. Mach. Learn. Res. 12, 2493–2537 (2011)MATH
5.
Zurück zum Zitat He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016-December, pp. 770–778 (2016) He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016-December, pp. 770–778 (2016)
6.
Zurück zum Zitat Vaswani, A., et al.: Attention is all you need. Advance Neural Information Processing Systems 2017-December, pp. 5999–6009 (2017) Vaswani, A., et al.: Attention is all you need. Advance Neural Information Processing Systems 2017-December, pp. 5999–6009 (2017)
7.
Zurück zum Zitat Howard, J., Ruder, S.: Universal language model fine-tuning for text classification. In: ACL 2018 - 56th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Long Papers), pp. 328–339 (2018) Howard, J., Ruder, S.: Universal language model fine-tuning for text classification. In: ACL 2018 - 56th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Long Papers), pp. 328–339 (2018)
8.
Zurück zum Zitat Peters, M.E., et al.: Deep contextualized word representations. In: NAACL HLT 2018 - 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies - Proceedings of the Conference, pp. 2227–2237 (2018) Peters, M.E., et al.: Deep contextualized word representations. In: NAACL HLT 2018 - 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies - Proceedings of the Conference, pp. 2227–2237 (2018)
9.
Zurück zum Zitat Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: NAACL HLT 2019 - 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies - Proceedings of the Conference, pp. 4171–4186 (2019) Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: NAACL HLT 2019 - 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies - Proceedings of the Conference, pp. 4171–4186 (2019)
10.
Zurück zum Zitat Liu, Y., et al.: RoBERTa: a robustly optimized BERT pretraining approach. arXiv (2019) Liu, Y., et al.: RoBERTa: a robustly optimized BERT pretraining approach. arXiv (2019)
11.
Zurück zum Zitat Zhang, Y., et al.: Dialogpt : large-scale generative pre-training for conversational response generation. arXiv, pp. 270–278 (2019) Zhang, Y., et al.: Dialogpt : large-scale generative pre-training for conversational response generation. arXiv, pp. 270–278 (2019)
12.
Zurück zum Zitat Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., Soricut, R.: Albert: a lite BERT for self-supervised learning of language representations. arXiv, pp. 1–17 (2019) Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., Soricut, R.: Albert: a lite BERT for self-supervised learning of language representations. arXiv, pp. 1–17 (2019)
13.
Zurück zum Zitat Raffel, C., et al.: Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv 21, 1–67 (2019) Raffel, C., et al.: Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv 21, 1–67 (2019)
14.
Zurück zum Zitat Petrock, V.: US Voice Assistant Users 2019 Who, What, When, Where and Why (2019) Petrock, V.: US Voice Assistant Users 2019 Who, What, When, Where and Why (2019)
15.
Zurück zum Zitat Edunov, S., Ott, M., Auli, M., Grangier, D.: Understanding back-translation at scale. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP 2018 (2020) Edunov, S., Ott, M., Auli, M., Grangier, D.: Understanding back-translation at scale. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP 2018 (2020)
Metadaten
Titel
Using BERT Model for Intent Classification in Human-Computer Dialogue Systems to Reduce Data Volume Requirement
verfasst von
Hao Liu
Huaming Peng
Copyright-Jahr
2021
DOI
https://doi.org/10.1007/978-3-030-80285-1_59

Premium Partner