Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Published in:
A Multi-objective Optimization Algorithm Based on Preference Three-Way Decomposition Read chapter Read first chapter

2018 | OriginalPaper | Chapter

A Hybrid RNN-CNN Encoder for Neural Conversation Model

Authors : Zhiyuan Ma, Wenge Rong, Yanmeng Wang, Libin Shi, Zhang Xiong

Published in: Knowledge Science, Engineering and Management

Publisher: Springer International Publishing

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

The conventional dialogue system is retrieval-based and its performance is directly limited by the size of dataset. Such dialogue system will give improper response if the question is out of dataset. Recently, due to the successful application of neural network in machine translation, the attention is diverted into building generative dialogue system using sequence to sequence (seq2seq) learning with neural networks. However, it is still difficult to build a satisfactory neural conversation model as sometimes the system tends to generate a general response. Nowadays, the widely employed method for dialogue generation is neural conversation model whose main structure is composed by a recurrent neural networks (RNNs) encoder-decoder. It is noticed that there is still a little work to introduce convolutional neural networks (CNNs) to neural conversation model. Considering that CNN has been used in many natural language processing (NLP) tasks and achieves great improvements, in this research we try to improve the performance of the neural conversation model by introducing a hybrid RNN-CNN encoder. The experimental result shows this architecture’s promising potential.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
previous chapter Constructive Justification Extraction for OWL Ontologies
next chapter Cross-Dataset Person Re-identification Using Similarity Preserved Generative Adversarial Networks
Footnotes
Literature
1.
2.
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. CoRR abs/1409.0473 (2014)
3.
Cho, K., van Merrienboer, B., Bahdanau, D., Bengio, Y.: On the properties of neural machine translation: encoder-decoder approaches. In: Proceedings of 8th Workshop on Syntax, Semantics and Structure in Statistical Translation, pp. 103–111 (2014)
4.
Cho, K., et al.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, pp. 1724–1734 (2014)
5.
Chung, J., Gülçehre, Ç., Cho, K., Bengio, Y.: Empirical evaluation of gated recurrent neural networks on sequence modeling. CoRR abs/1412.3555 (2014)
6.
Gehring, J., Auli, M., Grangier, D., Yarats, D., Dauphin, Y.N.: Convolutional sequence to sequence learning. In: Proceedings of the 34th International Conference on Machine Learning, vol. 70, pp. 1243–1252 (2017)
7.
Goodfellow, I.J., et al.: Generative adversarial nets. In: Proceedings of 2014 Annual Conference on Neural Information Processing Systems, pp. 2672–2680 (2014)
8.
Isbell Jr., C.L., Kearns, M., Kormann, D., Singh, S., Stone, P.: Cobot in LambdaMOO: a social statistics agent. In: Proceedings of the 17th National Conference on Artificial Intelligence and 12th Conference on on Innovative Applications of Artificial Intelligence, pp. 36–41 (2000)
9.
Kim, Y.: Convolutional neural networks for sentence classification. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, pp. 1746–1751 (2014)
10.
Li, J., Monroe, W., Ritter, A., Jurafsky, D., Galley, M., Gao, J.: Deep reinforcement learning for dialogue generation. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 1192–1202 (2016)
11.
Li, J., Monroe, W., Shi, T., Jean, S., Ritter, A., Jurafsky, D.: Adversarial learning for neural dialogue generation. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 2157–2169 (2017)
12.
Liu, C., Lowe, R., Serban, I., Noseworthy, M., Charlin, L., Pineau, J.: How not to evaluate your dialogue system: an empirical study of unsupervised evaluation metrics for dialogue response generation. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 2122–2132 (2016)
13.
Lowe, R., Pow, N., Serban, I., Pineau, J.: The ubuntu dialogue corpus: a large dataset for research in unstructured multi-turn dialogue systems. In: Proceedings of the 16th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pp. 285–294 (2015)
14.
Meng, F., Lu, Z., Wang, M., Li, H., Jiang, W., Liu, Q.: Encoding source language with convolutional neural network for machine translation. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, pp. 20–30 (2015)
15.
Mirza, M., Osindero, S.: Conditional generative adversarial nets. CoRR abs/1411.1784 (2014)
16.
Mou, L., Song, Y., Yan, R., Li, G., Zhang, L., Jin, Z.: Sequence to backward and forward sequences: a content-introducing approach to generative short-text conversation. In: Proceedings of 26th International Conference on Computational Linguistics, pp. 3349–3358 (2016)
17.
Prager, J.M., Brown, E.W., Coden, A., Radev, D.R.: Question-answering by predictive annotation. In: Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 184–191 (2000)
18.
Ritter, A., Cherry, C., Dolan, W.B.: Data-driven response generation in social media. In: Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pp. 583–593 (2011)
19.
Sennrich, R., Haddow, B., Birch, A.: Neural machine translation of rare words with subword units. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (2016)
20.
Serban, I.V., Sordoni, A., Bengio, Y., Courville, A.C., Pineau, J.: Building end-to-end dialogue systems using generative hierarchical neural network models. In: Proceedings of the 30th AAAI Conference on Artificial Intelligence, pp. 3776–3784 (2016)
21.
Shao, L., Gouws, S., Britz, D., Goldie, A., Strope, B., Kurzweil, R.: Generating long and diverse responses with neural conversation models. CoRR abs/1701.03185 (2017)
22.
Song, X., He, X., Gao, J., Deng, L.: Unsupervised learning of word semantic embedding using the deep structured semantic model. Microsoft Research (2014)
23.
Sordoni, A., et al.: A neural network approach to context-sensitive generation of conversational responses. In: Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 196–205 (2015)
24.
Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: Proceedings of 2014 Annual Conference on Neural Information Processing Systems 2014, pp. 3104–3112 (2014)
25.
Vinyals, O., Le, Q.V.: A neural conversational model. CoRR abs/1506.05869 (2015)
26.
Xing, C., et al.: Topic aware neural response generation. In: Proceedings of the 31st AAAI Conference on Artificial Intelligence, pp. 3351–3357 (2017)
27.
Yin, W., Schütze, H., Xiang, B., Zhou, B.: ABCNN: attention-based convolutional neural network for modeling sentence pairs. Trans. Assoc. Comput. Linguist. 4, 259–272 (2016)
28.
Yu, K., Zhao, Z., Wu, X., Lin, H., Liu, X.: Rich short text conversation using semantic-key-controlled sequence generation. IEEE/ACM Trans. Audio Speech Lang. Process. 26(8), 1359–1368 (2018)CrossRef
29.
Yu, L., Zhang, W., Wang, J., Yu, Y.: SeqGAN: sequence generative adversarial nets with policy gradient. In: Proceedings of the 31st AAAI Conference on Artificial Intelligence, pp. 2852–2858 (2017)
Metadata
Title
A Hybrid RNN-CNN Encoder for Neural Conversation Model
Authors
Zhiyuan Ma
Wenge Rong
Yanmeng Wang
Libin Shi
Zhang Xiong
Copyright Year
2018
Book
Knowledge Science, Engineering and Management

Print ISBN: 978-3-319-99246-4

Electronic ISBN: 978-3-319-99247-1

Copyright Year: 2018

DOI
https://doi.org/10.1007/978-3-319-99247-1_14

Premium Partner

    Image Credits