Skip to main content
Top

2022 | OriginalPaper | Chapter

Distill-AER: Fine-Grained Address Entity Recognition from Spoken Dialogue via Knowledge Distillation

Authors : Yitong Wang, Xue Han, Feng Zhou, Yiting Wang, Chao Deng, Junlan Feng

Published in: Natural Language Processing and Chinese Computing

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Fine-grained address entity recognition from spoken dialogue is an important but challenging task because there are multiple types of address entities distributed across the multi-round dialogue context. Existing work typically formulates this problem as a fine-grained named entity recognition task, which in our scenario suffers from a high cost of training data annotation. On the other hand, large-scale full standard addresses could be easily crawled from the web pages like Google Maps and annotated with fine-grained address tags with limited human effort. Leveraging this, we propose a distillation approach (Distill-AER) for transferring knowledge from the large-scale labeled full standard address dataset to the fine-grained address entity recognition task in a spoken dialogue context scenario. We further construct a labeled spoken dialogue dataset with address entities using the data augmentation paradigm we proposed, which could benefit future research. Experimental results show that Distill-AER significantly outperforms other competitive baselines.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Eligüzel, N., Çetinkaya, C., Dereli, T.: Comparison of different machine learning techniques on location extraction by utilizing geo-tagged tweets: a case study. Adv. Eng. Inform. 46, 101151 (2020)CrossRef Eligüzel, N., Çetinkaya, C., Dereli, T.: Comparison of different machine learning techniques on location extraction by utilizing geo-tagged tweets: a case study. Adv. Eng. Inform. 46, 101151 (2020)CrossRef
2.
go back to reference Yaman, E., Krdžalic-Koric, K.: Address entities extraction using named entity recognition. In: 2019 7th International Conference on Future Internet of Things and Cloud Workshops (FiCloudW), pp. 13–17. IEEE (2019) Yaman, E., Krdžalic-Koric, K.: Address entities extraction using named entity recognition. In: 2019 7th International Conference on Future Internet of Things and Cloud Workshops (FiCloudW), pp. 13–17. IEEE (2019)
3.
go back to reference Xu C, Li J, Luo X, et al. Dlocrl: A deep learning pipeline for fine-grained location recognition and linking in tweets. The World Wide Web Conference, pp. 3391–3397 (2019) Xu C, Li J, Luo X, et al. Dlocrl: A deep learning pipeline for fine-grained location recognition and linking in tweets. The World Wide Web Conference, pp. 3391–3397 (2019)
4.
go back to reference Zhang, H., Liu, L., Jiang, H., et al.: Texsmart: a text understanding system for fine-grained NER and enhanced semantic analysis. arXiv preprint arXiv:2012.15639 (2020) Zhang, H., Liu, L., Jiang, H., et al.: Texsmart: a text understanding system for fine-grained NER and enhanced semantic analysis. arXiv preprint arXiv:​2012.​15639 (2020)
5.
go back to reference Wu, G., Tang, G., Wang, Z., et al.: An attention-based BiLSTM-CRF model for Chinese clinic named entity recognition. IEEE Access 7, 113942–113949 (2019)CrossRef Wu, G., Tang, G., Wang, Z., et al.: An attention-based BiLSTM-CRF model for Chinese clinic named entity recognition. IEEE Access 7, 113942–113949 (2019)CrossRef
6.
go back to reference Schuster, M., Paliwal, K.K.: Bidirectional recurrent neural networks. IEEE Trans. Signal Process. 45(11), 2673–2681 (1997)CrossRef Schuster, M., Paliwal, K.K.: Bidirectional recurrent neural networks. IEEE Trans. Signal Process. 45(11), 2673–2681 (1997)CrossRef
7.
go back to reference Lafferty J, McCallum, A., Pereira, F.C.N.: Conditional random fields: probabilistic models for segmenting and labeling sequence data (2001) Lafferty J, McCallum, A., Pereira, F.C.N.: Conditional random fields: probabilistic models for segmenting and labeling sequence data (2001)
8.
go back to reference Mai, K., Pham, T.H., Nguyen, M.T., et al.: An empirical study on fine-grained named entity recognition. In: Proceedings of the 27th International Conference on Computational Linguistics, pp. 711–722 (2018) Mai, K., Pham, T.H., Nguyen, M.T., et al.: An empirical study on fine-grained named entity recognition. In: Proceedings of the 27th International Conference on Computational Linguistics, pp. 711–722 (2018)
9.
go back to reference Shan, S., Li, Z., Yang, Q., et al.: Geographical address representation learning for address matching. World Wide Web 23(3), 2005–2022 (2020)CrossRef Shan, S., Li, Z., Yang, Q., et al.: Geographical address representation learning for address matching. World Wide Web 23(3), 2005–2022 (2020)CrossRef
10.
go back to reference Chen, X., Qiu, X., Zhu, C., et al.: Gated recursive neural network for Chinese word segmentation. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 1744–1753 (2015) Chen, X., Qiu, X., Zhu, C., et al.: Gated recursive neural network for Chinese word segmentation. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 1744–1753 (2015)
11.
go back to reference Xu Y, Huang H, Feng C, et al.: A supervised multi-head self-attention network for nested named entity recognition. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 16, pp. 14185–14193 (2021) Xu Y, Huang H, Feng C, et al.: A supervised multi-head self-attention network for nested named entity recognition. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 16, pp. 14185–14193 (2021)
12.
go back to reference Misawa, S., Taniguchi, M., Miura, Y., et al.: Character-based bidirectional LSTM-CRF with words and characters for Japanese named entity recognition. In: Proceedings of the First Workshop on Subword and Character Level Models in NLP, pp. 97–102 (2017) Misawa, S., Taniguchi, M., Miura, Y., et al.: Character-based bidirectional LSTM-CRF with words and characters for Japanese named entity recognition. In: Proceedings of the First Workshop on Subword and Character Level Models in NLP, pp. 97–102 (2017)
13.
14.
go back to reference Lin, H., Lu, Y., Tang, J., et al.: A rigorous study on named entity recognition: can fine-tuning pretrained model lead to the promised land? arXiv preprint arXiv:2004.12126 (2020) Lin, H., Lu, Y., Tang, J., et al.: A rigorous study on named entity recognition: can fine-tuning pretrained model lead to the promised land? arXiv preprint arXiv:​2004.​12126 (2020)
15.
go back to reference Ju, M., Miwa, M., Ananiadou, S.: A neural layered model for nested named entity recognition. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pp. 1446–1459 (2018) Ju, M., Miwa, M., Ananiadou, S.: A neural layered model for nested named entity recognition. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pp. 1446–1459 (2018)
16.
go back to reference Monaikul, N., Castellucci, G., Filice, S., et al.: Continual learning for named entity recognition. In: Proceedings of the Thirty-Fifth AAAI Conference on Artificial Intelligence, pp. 13570–13577 (2021) Monaikul, N., Castellucci, G., Filice, S., et al.: Continual learning for named entity recognition. In: Proceedings of the Thirty-Fifth AAAI Conference on Artificial Intelligence, pp. 13570–13577 (2021)
17.
go back to reference Zhu, H., He, C., Fang, Y., et al.: Fine grained named entity recognition via seq2seq framework. IEEE Access 8, 53953–53961 (2020)CrossRef Zhu, H., He, C., Fang, Y., et al.: Fine grained named entity recognition via seq2seq framework. IEEE Access 8, 53953–53961 (2020)CrossRef
18.
go back to reference Li, P., Luo, A., Liu, J., et al.: Bidirectional gated recurrent unit neural network for Chinese address element segmentation. ISPRS Int. J. Geo Inf. 9(11), 635 (2020)CrossRef Li, P., Luo, A., Liu, J., et al.: Bidirectional gated recurrent unit neural network for Chinese address element segmentation. ISPRS Int. J. Geo Inf. 9(11), 635 (2020)CrossRef
19.
go back to reference Zhang, H., Ren, F., Li, H., et al.: Recognition method of new address elements in Chinese address matching based on deep learning. ISPRS Int. J. Geo Inf. 9(12), 745 (2020)CrossRef Zhang, H., Ren, F., Li, H., et al.: Recognition method of new address elements in Chinese address matching based on deep learning. ISPRS Int. J. Geo Inf. 9(12), 745 (2020)CrossRef
20.
go back to reference Kuai, X., Guo, R., Zhang, Z., et al.: Spatial context-based local toponym extraction and Chinese textual address segmentation from urban POI data. ISPRS Int. J. Geo Inf. 9(3), 147 (2020)CrossRef Kuai, X., Guo, R., Zhang, Z., et al.: Spatial context-based local toponym extraction and Chinese textual address segmentation from urban POI data. ISPRS Int. J. Geo Inf. 9(3), 147 (2020)CrossRef
21.
22.
go back to reference Sanh, V., Debut, L., Chaumond, J., et al.: DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108 (2019) Sanh, V., Debut, L., Chaumond, J., et al.: DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv preprint arXiv:​1910.​01108 (2019)
23.
go back to reference Jiao, X., Yin, Y., Shang, L., et al.: TinyBERT: distilling BERT for natural language understanding. In: Findings of the Association for Computational Linguistics: EMNLP 2020, pp. 4163–4174 (2020) Jiao, X., Yin, Y., Shang, L., et al.: TinyBERT: distilling BERT for natural language understanding. In: Findings of the Association for Computational Linguistics: EMNLP 2020, pp. 4163–4174 (2020)
24.
go back to reference Sun, Z., Yu, H., Song, X., et al.: Mobilebert: a compact task-agnostic Bert for resource-limited devices. arXiv preprint arXiv:2004.02984 (2020) Sun, Z., Yu, H., Song, X., et al.: Mobilebert: a compact task-agnostic Bert for resource-limited devices. arXiv preprint arXiv:​2004.​02984 (2020)
25.
go back to reference Tsai, H., Riesa, J., Johnson, M., et al.: Small and practical BERT models for sequence labeling. arXiv preprint arXiv:1909.00100 (2019) Tsai, H., Riesa, J., Johnson, M., et al.: Small and practical BERT models for sequence labeling. arXiv preprint arXiv:​1909.​00100 (2019)
26.
go back to reference Yang, H., Huang, S., Dai, X., et al.: Fine-grained knowledge fusion for sequence labeling domain adaptation. arXiv preprint arXiv:1909.04315 (2019) Yang, H., Huang, S., Dai, X., et al.: Fine-grained knowledge fusion for sequence labeling domain adaptation. arXiv preprint arXiv:​1909.​04315 (2019)
27.
go back to reference Orihashi, S., Yamazaki, Y., Makishima, N., et al.: Hierarchical knowledge distillation for dialogue sequence labeling. arXiv preprint arXiv:2111.10957 (2021) Orihashi, S., Yamazaki, Y., Makishima, N., et al.: Hierarchical knowledge distillation for dialogue sequence labeling. arXiv preprint arXiv:​2111.​10957 (2021)
28.
go back to reference Cui, Y., Che, W., Liu, T., et al.: Pre-training with whole word masking for Chinese Bert. IEEE/ACM Trans. Audio Speech Lang. Process. 29, 3504–3514 (2021)CrossRef Cui, Y., Che, W., Liu, T., et al.: Pre-training with whole word masking for Chinese Bert. IEEE/ACM Trans. Audio Speech Lang. Process. 29, 3504–3514 (2021)CrossRef
29.
go back to reference Chaudhary, A.: A visual survey of data augmentation in NLP (2020) Chaudhary, A.: A visual survey of data augmentation in NLP (2020)
30.
go back to reference Tian, X., Huang, X., He, D., et al.: TOD-DA: towards boosting the robustness of task-oriented dialogue modeling on spoken conversations. arXiv preprint arXiv:2112.12441 (2021) Tian, X., Huang, X., He, D., et al.: TOD-DA: towards boosting the robustness of task-oriented dialogue modeling on spoken conversations. arXiv preprint arXiv:​2112.​12441 (2021)
31.
Metadata
Title
Distill-AER: Fine-Grained Address Entity Recognition from Spoken Dialogue via Knowledge Distillation
Authors
Yitong Wang
Xue Han
Feng Zhou
Yiting Wang
Chao Deng
Junlan Feng
Copyright Year
2022
DOI
https://doi.org/10.1007/978-3-031-17120-8_50

Premium Partner