nach oben

Erschienen in:

2024 | OriginalPaper | Buchkapitel

A Comparative Study of Knowledge Graph-to-Text Generation Architectures in the Context of Conversational Agents

verfasst von : Hussam Ghanem, Christophe Cruz

Erschienen in: Complex Networks & Their Applications XII

Verlag: Springer Nature Switzerland

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

This work delves into the dynamic landscape of Knowledge Graph-to-text generation, where structured knowledge graphs are transformed into coherent natural language text. Three key architectural paradigms are explored: Graph Neural Networks (GNNs), Graph Transformers (GTs), and linearization with sequence-to-sequence models. We discuss the advantages and limitations of these architectures, and we do some experiments on these architectures. Performance evaluations on WebNLG V.2 demonstrate the superiority of sequence-to-sequence Transformer-based models, especially when enriched with structural information from the graph. Despite being unsupervised, the CycleGT model also outperforms GNNs and GTs. However, practical constraints, such as computational efficiency and model validity, make sequence-to-sequence models the preferred choice for real-time conversational agents. Future research directions include enhancing the efficiency of GNNs and GTs, addressing scalability issues, handling multimodal knowledge graphs, improving interpretability, and devising data labeling strategies for domain-specific models. Cross-lingual and multilingual extensions can further broaden the applicability of these models in diverse linguistic contexts. In conclusion, the choice of architecture should align with specific task requirements and application constraints, and the field offers promising prospects for continued innovation and refinement.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Vorheriges Kapitel L2G2G: A Scalable Local-to-Global Network Embedding with Graph Autoencoders

Nächstes Kapitel Network Embedding Based on DepDist Contraction

Balažević, I., Allen, C., Hospedales, T.M.: Tucker: tensor factorization for knowledge graph completion. arXiv preprint arXiv:1901.09590 (2019)

Banerjee, S., Lavie, A.: METEOR: an automatic metric for MT evaluation with improved correlation with human judgments. In: Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization (2005)

Bordes, A., et al.: Translating embeddings for modeling multi-relational data. In: Advances in Neural Information Processing Systems, vol. 26 (2013)

Cai, D., Lam, W.: Graph transformer for graph-to-sequence learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 05 (2020)

Chaudhri, V., et al.: Knowledge graphs: introduction, history and perspectives. AI Mag. 43(1), 17–29 (2022)

Chen, Y., Wu, L., Zaki, M.J.: Reinforcement learning based graph-to-sequence model for natural question generation. arXiv preprint arXiv:1908.04942 (2019)

Chen, Y., Wu, L., Zaki, M.J.: Toward subgraph-guided knowledge graph question generation with graph neural networks. IEEE Trans. Neural Netw. Learn. Syst., 1–12 (2023)

Chen, W., et al.: KGPT: knowledge-grounded pre-training for data-to-text generation. arXiv preprint arXiv:2010.02307 (2020)

Dai, Y., et al.: A survey on knowledge graph embedding: approaches, applications and benchmarks. Electronics 9(5), 750 (2020)CrossRef

10.

Distiawan, B., et al.: GTR-LSTM: a triple encoder for sentence generation from RDF data. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (2018)

11.

Duvenaud, D.K., et al.: Convolutional networks on graphs for learning molecular fingerprints. In: Advances in Neural Information Processing Systems, vol. 28 (2015)

12.

Ferreira, T.C., et al.: Enriching the WebNLG corpus. In: Proceedings of the 11th International Conference on Natural Language Generation (2018)

13.

Ferreira, T.C., et al.: Neural data-to-text generation: a comparison between pipeline and end-to-end architectures. arXiv preprint arXiv:1908.09022 (2019)

14.

Fu, Z., et al.: Partially-aligned data-to-text generation with distant supervision. arXiv preprint arXiv:2010.01268 (2020)

15.

Gardent, C., et al.: The WebNLG challenge: generating text from RDF data. In: Proceedings of the 10th International Conference on Natural Language Generation (2017)

16.

Gilmer, J., et al.: Neural message passing for quantum chemistry. In: International Conference on Machine Learning. PMLR (2017)

17.

Guo, Q., et al.: CycleGT: unsupervised graph-to-text and text-to-graph generation via cycle training. arXiv preprint arXiv:2006.04702 (2020)

18.

Guo, Q., et al.: Fork or fail: cycle-consistent training with many-to-one mappings. In: International Conference on Artificial Intelligence and Statistics. PMLR (2021)

19.

Guo, Q., et al.: P2: a plan-and-pretrain approach for knowledge graph-to-text generation: a plan-and-pretrain approach for knowledge graph-to-text generation. In: Proceedings of the 3rd International Workshop on Natural Language Generation from the Semantic Web (WebNLG+) (2020)

20.

Guo, Z., et al.: Densely connected graph convolutional networks for graph-to-sequence learning. Trans. Assoc. Comput. Linguist. 7, 297–312 (2019)CrossRef

21.

Hogan, A., et al.: Knowledge graphs. ACM Comput. Surv. (Csur) 54(4), 1–37 (2021)

22.

Joshi, C.: Transformers are graph neural networks. The Gradient 7 (2020)

23.

Ke, P., et al.: JointGT: graph-text joint representation learning for text generation from knowledge graphs. arXiv preprint arXiv:2106.10502 (2021)

24.

Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016)

25.

Koncel-Kedziorski, R., et al.: Text generation from knowledge graphs with graph transformers. arXiv preprint arXiv:1904.02342 (2019)

26.

Lewis, M., et al.: Bart: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.13461 (2019)

27.

Li, J., et al.: Pretrained language models for text generation: A survey. arXiv preprint arXiv:2201.05273 (2022)

28.

Li, Y., et al.: Gated graph sequence neural networks. arXiv preprint arXiv:1511.05493 (2015)

29.

Liang, X., Shen, X., Feng, J., Lin, L., Yan, S.: Semantic object parsing with graph LSTM. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016, Part I. LNCS, vol. 9905, pp. 125–143. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_8CrossRef

30.

Lin, C.-Y.: Rouge: a package for automatic evaluation of summaries. Text summarization branches out (2004)

31.

Liu, L., et al.: How to train your agent to read and write. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 15 (2021)

32.

Liu, W., et al.: K-BERT: enabling language representation with knowledge graph. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 03 (2020)

33.

Mager, M., et al.: GPT-too: a language-model-first approach for AMR-to-text generation. arXiv preprint arXiv:2005.09123 (2020)

34.

Marcheggiani, D., Frolov, A., Titov, I.: A simple and accurate syntax-agnostic neural model for dependency-based semantic role labeling. arXiv preprint arXiv:1701.02593 (2017)

35.

Moon, S., et al.: OpenDialKG: explainable conversational reasoning with attention-based walks over knowledge graphs. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (2019)

36.

Moryossef, A., Goldberg, Y., Dagan, I.: Step-by-step: separating planning from realization in neural data-to-text generation. arXiv preprint arXiv:1904.03396 (2019)

37.

Nan, L., et al.: DART: Open-domain structured data record to text generation. arXiv preprint arXiv:2007.02871 (2020)

38.

Noy, N., et al.: Industry-scale Knowledge Graphs: Lessons and Challenges: Five diverse technology companies show how it’s done. Queue 17(2), 48–75 (2019)CrossRef

39.

Papineni, K., et al.: Bleu: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (2002)

40.

Peters, M.E., et al.: Knowledge enhanced contextual word representations. arXiv preprint arXiv:1909.04164 (2019)

41.

Puduppully, R., Dong, L., Lapata, M.: Data-to-text generation with content selection and planning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33. no. 01 (2019)

42.

Puzikov, Y., Gurevych, I.: E2E NLG challenge: neural models vs. templates. In: Proceedings of the 11th International Conference on Natural Language Generation (2018)

43.

Radford, A., et al.: Language models are unsupervised multitask learners. OpenAI blog 1(8), 9 (2019)

44.

Raffel, C., et al.: Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res. 21(1), 5485–5551 (2020)MathSciNet

45.

Ribeiro, L.F.R., et al.: Investigating pretrained language models for graph-to-text generation. arXiv preprint arXiv:2007.08426 (2020)

46.

Ribeiro, L.F.R., et al.: Modeling global and local node contexts for text generation from knowledge graphs. Trans. Assoc. Comput. Linguist. 8, 589–604 (2020)CrossRef

47.

Scarselli, F., et al.: The graph neural network model. IEEE Trans. Neural Netw. 20(1), 61–80 (2008)

48.

Schmitt, M., et al.: Modeling graph structure via relative position for text generation from knowledge graphs. arXiv preprint arXiv:2006.09242 (2020)

49.

Shimorina, A., Gardent, C.: Handling rare items in data-to-text generation. In: Proceedings of the 11th International Conference on Natural Language Generation (2018)

50.

Song, L., et al.: A graph-to-sequence model for AMR-to-text generation. arXiv preprint arXiv:1805.02473 (2018)

51.

Tai, K.S., Socher, R., Manning, C.D.: Improved semantic representations from tree-structured long short-term memory networks. arXiv preprint arXiv:1503.00075 (2015)

52.

Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)

53.

Vedantam, R., Lawrence Zitnick, C., Parikh, D.: Cider: consensus-based image description evaluation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2015)

54.

Veličković, P., et al.: Graph attention networks. arXiv preprint arXiv:1710.10903 (2017)

55.

Wahde, M., Virgolin, M.: Conversational agents: theory and applications. In: Handbook on Computer Learning and Intelligence: Volume 2: Deep Learning, Intelligent Control and Evolutionary Computation, pp. 497–544 (2022)

56.

Wang, Z., Zhang, J., Feng, J., Chen, Z.: Knowledge graph embedding by translating on hyperplanes. In: Proceedings of the 28th AAAI Conference on Artificial Intelligence, pp. 1112–1119. AAAI Press (2014)

57.

Zhang, Z., et al.: ERNIE: enhanced language representation with informative entities. arXiv preprint arXiv:1905.07129 (2019)

58.

Zhang, Y., et al.: Meta-transformer: a unified framework for multimodal learning. arXiv preprint arXiv:2307.10802 (2023)

59.

Zhao, C., Walker, M., Chaturvedi, S.: Bridging the structural gap between encoding and decoding for data-to-text generation. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (2020)

Titel: A Comparative Study of Knowledge Graph-to-Text Generation Architectures in the Context of Conversational Agents
verfasst von: Hussam Ghanem
Christophe Cruz
Verlag: Springer Nature Switzerland
Buch: Complex Networks & Their Applications XII
Print ISBN: 978-3-031-53467-6

Electronic ISBN: 978-3-031-53468-3

Copyright-Jahr: 2024
DOI: https://doi.org/10.1007/978-3-031-53468-3_35

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partner