Skip to main content

2023 | OriginalPaper | Buchkapitel

Research and Application Status of Text Generation Tasks Based on Generative Adversarial Network

verfasst von : Weiqi Wang, Dan Jiang, Shaozhong Cao

Erschienen in: IEIS 2022

Verlag: Springer Nature Singapore

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

In recent years, in the field of natural language processing, significant progress has been made in text generation. Text generation has gained widespread popularity in many fields such as abstract extraction, poetry creation, and response to social network comments. Given the excellent generative capabilities of Generative Adversarial Networks (GAN), it is often used as main model for text generation with remarkable results. This review aims to provide the core tasks of generative adversarial network text generation and the architecture used to deal with these tasks, and draw attention to the challenges in text generation with generative adversarial network. Firstly, we outline the mainstream text generation models, and then introduce datasets, advanced models and challenges of text generation tasks in detail. Finally, we discuss the prospects and challenges of the fusion of generative adversarial networks and text generation tasks in the future.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Kalchbrenner, N., Blunsom, P.: Recurrent continuous translation models. In: Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp.1700–1709 (2013) Kalchbrenner, N., Blunsom, P.: Recurrent continuous translation models. In: Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp.1700–1709 (2013)
2.
Zurück zum Zitat Gehring, J., Auli, M., Grangier, D., et al.: A convolutional encoder model for neural machine translation (2016) Gehring, J., Auli, M., Grangier, D., et al.: A convolutional encoder model for neural machine translation (2016)
3.
Zurück zum Zitat Zhen, Y., Wei, C., Feng, W., et al.: Improving neural machine translation with conditional sequence generative adversarial nets (2018) Zhen, Y., Wei, C., Feng, W., et al.: Improving neural machine translation with conditional sequence generative adversarial nets (2018)
4.
Zurück zum Zitat Cho, K., Merrienboer, B.V., Gulcehre, C., et al.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. Computer Science (2014) Cho, K., Merrienboer, B.V., Gulcehre, C., et al.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. Computer Science (2014)
5.
Zurück zum Zitat Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. Computer Science (2014) Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. Computer Science (2014)
6.
Zurück zum Zitat Cho, K., Merrienboer, B.V., Bahdanau, D., et al.: On the properties of neural machine translation: encoder-decoder approaches. Computer Science (2014) Cho, K., Merrienboer, B.V., Bahdanau, D., et al.: On the properties of neural machine translation: encoder-decoder approaches. Computer Science (2014)
7.
Zurück zum Zitat Li, J., Monroe, W., Shi, T., et al.: Adversarial learning for neural dialogue generation. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (2017) Li, J., Monroe, W., Shi, T., et al.: Adversarial learning for neural dialogue generation. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (2017)
8.
Zurück zum Zitat Weizenbaum, J.: ELIZA: a computer program for the study of natural language communication between men and machines (1974) Weizenbaum, J.: ELIZA: a computer program for the study of natural language communication between men and machines (1974)
9.
Zurück zum Zitat Colby, K.M.: Artificial paranoia: a computer simulation of paranoid processes. Elsevier (2013) Colby, K.M.: Artificial paranoia: a computer simulation of paranoid processes. Elsevier (2013)
10.
Zurück zum Zitat Shieber, S.M.: Lessons from a restricted Turing test. Commun. ACM 37, 70–78 (1994) Shieber, S.M.: Lessons from a restricted Turing test. Commun. ACM 37, 70–78 (1994)
12.
Zurück zum Zitat Rush, A.M., Chopra, S., Weston, J. : A neural attention model for abstractive sentence summarization. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 379–389 (2015) Rush, A.M., Chopra, S., Weston, J. : A neural attention model for abstractive sentence summarization. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 379–389 (2015)
13.
Zurück zum Zitat Zeng, W., Luo, W., Fidler, S., et al.: Efficient summarization with read-again and copy mechanism. arXiv preprint arXiv:1611.03382 (2016) Zeng, W., Luo, W., Fidler, S., et al.: Efficient summarization with read-again and copy mechanism. arXiv preprint arXiv:​1611.​03382 (2016)
14.
Zurück zum Zitat Nallapati, R., et al.: Abstractive text summarization using sequence-to-sequence RNNs and beyond. In: Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning (2016) Nallapati, R., et al.: Abstractive text summarization using sequence-to-sequence RNNs and beyond. In: Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning (2016)
15.
Zurück zum Zitat Rush, A.M., Chopra, S., Weston, J.: A Neural attention model for abstractive sentence summarization. Computer Science (2015) Rush, A.M., Chopra, S., Weston, J.: A Neural attention model for abstractive sentence summarization. Computer Science (2015)
16.
Zurück zum Zitat Alifimoff, A.: Abstractive sentence summarization with attentive deep recurrent neural networks (2016) Alifimoff, A.: Abstractive sentence summarization with attentive deep recurrent neural networks (2016)
17.
Zurück zum Zitat Prabhumoye, S., Tsvetkov, Y., Salakhutdinov, R., et al.: Style transfer through back-translation. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (2018) Prabhumoye, S., Tsvetkov, Y., Salakhutdinov, R., et al.: Style transfer through back-translation. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (2018)
18.
Zurück zum Zitat Zhu, J.-Y., Park, T., Isola, P., et al.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2223–2232 (2017) Zhu, J.-Y., Park, T., Isola, P., et al.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2223–2232 (2017)
19.
Zurück zum Zitat Zadeh, L.A.: From search engines to question-answering systems - the need for new tools. In: The 12th IEEE International Conference on Fuzzy Systems, 2003, pp. 1107–1109. FUZZ 2003. IEEE (2003) Zadeh, L.A.: From search engines to question-answering systems - the need for new tools. In: The 12th IEEE International Conference on Fuzzy Systems, 2003, pp. 1107–1109. FUZZ 2003.  IEEE (2003)
20.
Zurück zum Zitat Zhao, T., Ran, Z., Eskenazi, M.: Learning discourse-level diversity for neural dialog models using conditional variational autoencoders. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (2017) Zhao, T., Ran, Z., Eskenazi, M.: Learning discourse-level diversity for neural dialog models using conditional variational autoencoders. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (2017)
21.
Zurück zum Zitat Bengio, Y., Ducharme, R., Vincent, P., et al.: A neural probabilistic language model. J. Mach. Learn. Res. 3, 1137–1155 (2003) Bengio, Y., Ducharme, R., Vincent, P., et al.: A neural probabilistic language model. J. Mach. Learn. Res. 3, 1137–1155 (2003)
22.
Zurück zum Zitat Karpathy, A.: The unreasonable effectiveness of recurrent neural networks. Andrej Karpathy Blog 21, 23 (2015) Karpathy, A.: The unreasonable effectiveness of recurrent neural networks. Andrej Karpathy Blog 21, 23 (2015)
26.
Zurück zum Zitat Devlin, J., Chang, M.W., Lee, K., et al.: BERT: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018) Devlin, J., Chang, M.W., Lee, K., et al.: BERT: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:​1810.​04805 (2018)
27.
Zurück zum Zitat Yang, Z., Dai, Z., Yang, Y., et al.: XLNet: generalized autoregressive pretraining for language understanding. In: Proceedings of the International Conference on Neural Information Processing Systems, pp. 5753–5763 (2019) Yang, Z., Dai, Z., Yang, Y., et al.: XLNet: generalized autoregressive pretraining for language understanding. In: Proceedings of the International Conference on Neural Information Processing Systems, pp. 5753–5763 (2019)
28.
Zurück zum Zitat Radford, A., Wu, J., Child, R., et al.: Language models are unsupervised multitask learners. OpenAI Blog 1(8), 9 (2019) Radford, A., Wu, J., Child, R., et al.: Language models are unsupervised multitask learners. OpenAI Blog 1(8), 9 (2019)
29.
Zurück zum Zitat Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., et al.: Generative adversarial nets. In: Proceedings of the International Conference on Neural Information Processing Systems, pp. 2672–2680 (2014) Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., et al.: Generative adversarial nets. In: Proceedings of the International Conference on Neural Information Processing Systems, pp. 2672–2680 (2014)
30.
Zurück zum Zitat Neal, R.M.: Markov chain sampling methods for Dirichlet process mixture models. J. Comput. Graph. Stat. 9(2), 249–265 (2000) Neal, R.M.: Markov chain sampling methods for Dirichlet process mixture models. J. Comput. Graph. Stat. 9(2), 249–265 (2000)
31.
Zurück zum Zitat Guo, H.: Generating text with deep reinforcement learning. Comput. Sci. 40(4), 1–5 (2015) Guo, H.: Generating text with deep reinforcement learning. Comput. Sci. 40(4), 1–5 (2015)
32.
Zurück zum Zitat Martin, A.S.C., Bottou, L.: Wasserstein generative adversarial networks. In: Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia, pp. 214–223 (2017) Martin, A.S.C., Bottou, L.: Wasserstein generative adversarial networks. In: Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia, pp. 214–223 (2017)
33.
Zurück zum Zitat Kusner, M.J., Hernández-Lobato, J.M.: GANs for sequences of discrete elements with the gumbel-softmax distribution. arXiv preprint arXiv:1611.04051 (2016) Kusner, M.J., Hernández-Lobato, J.M.: GANs for sequences of discrete elements with the gumbel-softmax distribution. arXiv preprint arXiv:​1611.​04051 (2016)
34.
Zurück zum Zitat Zhang, Y., Gan, Z., Carin, L.: Generating text via adversarial training. In: NIPS workshop on Adversarial Training (2016) Zhang, Y., Gan, Z., Carin, L.: Generating text via adversarial training. In: NIPS workshop on Adversarial Training (2016)
35.
Zurück zum Zitat Fedus, W., Goodfellow, I., Dai, A.M.: MaskGAN: better Text Generation via Filling in the______ (2018) Fedus, W., Goodfellow, I., Dai, A.M.: MaskGAN: better Text Generation via Filling in the______ (2018)
36.
Zurück zum Zitat Xu, J., Ren, X., Lin, J., et al.: DP-GAN: diversity-promoting generative adversarial network for generating informative and diversified text (2018) Xu, J., Ren, X., Lin, J., et al.: DP-GAN: diversity-promoting generative adversarial network for generating informative and diversified text (2018)
37.
Zurück zum Zitat Nie, W., Narodytska, N., Patel, A.: RelGAN: relational generative adversarial networks for text generation. In: ICLR 2019 Conference Paper 1462 (2019) Nie, W., Narodytska, N., Patel, A.: RelGAN: relational generative adversarial networks for text generation. In: ICLR 2019 Conference Paper 1462 (2019)
38.
Zurück zum Zitat Yu, L., Zhang, W., Wang, J., et al. SeqGAN: sequence generative adversarial nets with policy gradient (2016) Yu, L., Zhang, W., Wang, J., et al. SeqGAN: sequence generative adversarial nets with policy gradient (2016)
39.
Zurück zum Zitat Sehnke, F., Osendorfer, C., Rückstieß, T., et al.: Parameter-exploring policy gradients. Neural Netw. 23(4), 551–559 (2010)CrossRef Sehnke, F., Osendorfer, C., Rückstieß, T., et al.: Parameter-exploring policy gradients. Neural Netw. 23(4), 551–559 (2010)CrossRef
40.
Zurück zum Zitat Williams, R.J.: Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach. Learn. 8(3–4), 229–256 (1992)CrossRef Williams, R.J.: Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach. Learn. 8(3–4), 229–256 (1992)CrossRef
41.
Zurück zum Zitat Che, T., Li, Y., Zhang, R., et al.: Maximum-likelihood augmented discrete generative adversarial networks. arXiv preprint arXiv:1702.07983 (2017) Che, T., Li, Y., Zhang, R., et al.: Maximum-likelihood augmented discrete generative adversarial networks. arXiv preprint arXiv:​1702.​07983 (2017)
42.
Zurück zum Zitat Guo, J., Lu, S., Han, C., et al.: Long text generation via adversarial training with leaked information (2017) Guo, J., Lu, S., Han, C., et al.: Long text generation via adversarial training with leaked information (2017)
43.
Zurück zum Zitat Vezhnevets, A.S., Osindero, S., Schaul, T., et al.: FeUdal networks for hierarchical reinforcement learning (2017) Vezhnevets, A.S., Osindero, S., Schaul, T., et al.: FeUdal networks for hierarchical reinforcement learning (2017)
44.
Zurück zum Zitat Kim, Y.: Convolutional neural networks for sentence classification. Eprint Arxiv (2014) Kim, Y.: Convolutional neural networks for sentence classification. Eprint Arxiv (2014)
45.
Zurück zum Zitat Mou, L., Yan, R., Li, G., et al.: Backward and forward language modeling for constrained sentence generation. Comput. Sci. 4(6), 473–482 (2015) Mou, L., Yan, R., Li, G., et al.: Backward and forward language modeling for constrained sentence generation. Comput. Sci. 4(6), 473–482 (2015)
46.
Zurück zum Zitat Sutton, R.S., Mcallester, D., Singh, S., et al.: Policy gradient methods for reinforcement learning with function approximation. MIT Press (1999) Sutton, R.S., Mcallester, D., Singh, S., et al.: Policy gradient methods for reinforcement learning with function approximation. MIT Press (1999)
47.
Zurück zum Zitat Dietterich, T.G.: An overview of MAXQ hierarchical reinforcement learning. In: Choueiry, B.Y., Walsh, T. (eds.) Abstraction, Reformulation, and Approximation. SARA 2000. Lecture Notes in Computer Science, vol. 1864. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-44914-0_2 Dietterich, T.G.: An overview of MAXQ hierarchical reinforcement learning. In: Choueiry, B.Y., Walsh, T. (eds.) Abstraction, Reformulation, and Approximation. SARA 2000. Lecture Notes in Computer Science, vol. 1864. Springer, Berlin, Heidelberg. https://​doi.​org/​10.​1007/​3-540-44914-0_​2
48.
Zurück zum Zitat Li, J., Luong, M.T., Dan, J.: A hierarchical neural autoencoder for paragraphs and documents. Computer Science (2015) Li, J., Luong, M.T., Dan, J.: A hierarchical neural autoencoder for paragraphs and documents. Computer Science (2015)
49.
Zurück zum Zitat Luong, T., Pham, H., Manning, C.D.: Effective approaches to attention-based neural machine translation. EMNLP 2015, 1412–1421 (2015) Luong, T., Pham, H., Manning, C.D.: Effective approaches to attention-based neural machine translation. EMNLP 2015, 1412–1421 (2015)
50.
Zurück zum Zitat Zhou, W., Ge, T., Xu, K., et al.: Self-adversarial learning with comparative discrimination for text generation (2020) Zhou, W., Ge, T., Xu, K., et al.: Self-adversarial learning with comparative discrimination for text generation (2020)
51.
Zurück zum Zitat Zhu, Y., Lu, S., Zheng, L., et al.: Texygen: A benchmarking platform for text generation models. In: Proceedings of the 41st International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1097–1100 (2018) Zhu, Y., Lu, S., Zheng, L., et al.: Texygen: A benchmarking platform for text generation models. In: Proceedings of the 41st International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1097–1100 (2018)
52.
Zurück zum Zitat Papineni, K., Roukos, S., Ward, T., et al.: BLEU: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pp. 311–318 (2002) Papineni, K., Roukos, S., Ward, T., et al.: BLEU: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pp. 311–318 (2002)
Metadaten
Titel
Research and Application Status of Text Generation Tasks Based on Generative Adversarial Network
verfasst von
Weiqi Wang
Dan Jiang
Shaozhong Cao
Copyright-Jahr
2023
Verlag
Springer Nature Singapore
DOI
https://doi.org/10.1007/978-981-99-3618-2_11

Premium Partner