Skip to main content
Fachgebiete
Automobil + Motoren Bauwesen + Immobilien Business IT + Informatik Elektrotechnik + Elektronik Energie + Nachhaltigkeit Finance + Banking Management + Führung Marketing + Vertrieb Maschinenbau + Werkstoffe Versicherung + Risiko
DE
EN
Bücher
Zeitschriften
Themenseiten
Organisationspsychologie Projektmanagement Marketing Smart Manufacturing
Weitere Formate
Podcasts Webinare Technik Webinare Wirtschaft Kongresse Veranstaltungskalender Awards
Jetzt Einzelzugang starten
Zugang für Unternehmen
Referenzkunden
SLX-Digitalkonferenz: Zukunftswerkstatt 2024

Mehr Umsatz mit Top-Kontakten

Am 7. Mai erfahren Sie, wie Vertriebsteams Leadgenerierung, Leadnurturing und Leadstrategien effizient steuern können.
Erweiterte Suche
Anmelden
nach oben
Neural Processing Letters
Erschienen in: Neural Processing Letters 6/2022

11.05.2022

Implicit Relation Inference with Deep Path Extraction for Commonsense Question Answering

verfasst von: Peng Yang, Zijian Liu, Bing Li, Penghui Zhang

Erschienen in: Neural Processing Letters | Ausgabe 6/2022

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Natural language inference plays an essential role in Commonsense Question Answering. Conventional models usually adopt keywords in questions and choices as queries to retrieve static and explicit evidence that is used to obtain final answers, where dynamic interaction between different keywords and implicit relations inference of deeper information are often neglected. In this paper, we propose a novel joint model, the Graph Relation retrieval Reasoning Network (GRRN), to explicitly introduce the dynamic interaction among different keywords and generate informative features that contribute to representation updating. In addition, to pursue in-depth relations between different keywords, we develop an optimised Path Evidence Fusion in the GRRN to obtain evidence based on deep paths and implicit relations with comprehensive knowledge by making full use of the original paths in external knowledge graphs. The experimental results show that compared with the baselines, our approach achieves remarkable improvement of 1.74\(\%\) for precision on the CommonsenseQA dataset, thereby demonstrating the superiority of our state-of-the-art approach on implicit relation inference with deep paths.
Vorheriger Artikel COLAM: Co-Learning of Deep Neural Networks and Soft Labels via Alternating Minimization
Nächster Artikel Color Image Enhancement: A Metaheuristic Chimp Optimization Algorithm

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Jetzt informieren
Fußnoten
Literatur
1.
Battaglia PW, Hamrick JB, Bapst V, Sanchez-Gonzalez A, Zambaldi V, Malinowski M, Tacchetti A, Raposo D, Santoro A, Faulkner R, et al. (2018) Relational inductive biases, deep learning, and graph networks. arXiv preprint arXiv:​1806.​01261
2.
Bordes A, Usunier N, Garcia-Durán A, Weston J, Yakhnenko O (2013) Translating embeddings for modeling multi-relational data. In: Proceedings of the 26th International Conference on Neural Information Processing Systems, vol 2, pp 2787–2795
3.
Bosselut A, Rashkin H, Sap M, Malaviya C, Celikyilmaz A, Choi Y (2019) Comet: Commonsense transformers for automatic knowledge graph construction. In: Proceedings of the 57th annual meeting of the association for computational linguistics, pp 4762–4779
4.
Bosselut A, Le Bras R, Choi Y (2021) Dynamic neuro-symbolic knowledge graph construction for zero-shot commonsense question answering. Proceedings of the AAAI Conference on Artificial Intelligence 35:4923–4931CrossRef
5.
Chen D, Fisch A, Weston J, Bordes A (2017) Reading wikipedia to answer open-domain questions. In: Proceedings of the 55th annual meeting of the association for computational linguistics, vol 1, pp 1870–1879
6.
Chen Q, Ji F, Chen H, Zhang Y (2020) Improving commonsense question answering by graph-based iterative retrieval over multiple knowledge sources. arXiv preprint arXiv:​2011.​02705
7.
Cho K, van Merrienboer B, Gülçehre Ç, Bahdanau D, Bougares F, Schwenk H, Bengio Y (2014) Learning phrase representations using rnn encoder-decoder for statistical machine translation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp 1724–1734
8.
Clark K, Luong MT, Le QV, Manning CD (2020) Electra: Pre-training text encoders as discriminators rather than generators. arXiv preprint arXiv:​2003.​10555
9.
Cui W, Xiao Y, Wang H, Song Y, Hwang Sw, Wang W (2019) Kbqa: learning question answering over qa corpora and knowledge bases. arXiv preprint arXiv:​1903.​02419
10.
Devlin J, Chang MW, Lee K, Toutanova K (2018) Bert: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:​1810.​04805
11.
Doxolodeo K, Mahendra R (2020) Ui at semeval-2020 task 4: Commonsense validation and explanation by exploiting contradiction. In: Proceedings of the Fourteenth Workshop on Semantic Evaluation, pp 614–619
12.
Fadel A, Al-Ayyoub M, Cambria E (2020) Justers at semeval-2020 task 4: Evaluating transformer models against commonsense validation and explanation. In: Proceedings of the fourteenth workshop on semantic evaluation, pp 535–542
13.
Feng Y, Chen X, Lin BY, Wang P, Yan J, Ren X (2020) Scalable multi-hop relational reasoning for knowledge-aware question answering. arXiv preprint arXiv:​2005.​00646
14.
Gretz S, Bilu Y, Cohen-Karlik E, Slonim N (2020) The workweek is the best time to start a family–a study of gpt-2 based claim generation. arXiv preprint arXiv:​2010.​06185
15.
Huang Y, Fang M, Zhan X, Cao Q, Liang X, Lin L (2021) Rem-net: Recursive erasure memory network for commonsense evidence refinement. Proceedings of the AAAI Conference on Artificial Intelligence 35:6375–6383CrossRef
16.
He X, Liu Q, Yang Y (2020) Mv-gnn: multi-view graph neural network for compression artifacts reduction. IEEE Trans Image Process 29:6829–6840CrossRefMATH
17.
Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780CrossRef
18.
Huang X, Zhang J, Li D, Li P (2019) Knowledge graph embedding based question answering. In: Proceedings of the twelfth ACM international conference on web search and data mining, pp 105–113
19.
Ji S, Pan S, Cambria E, Marttinen P, Yu PS (2022) A survey on knowledge graphs: Representation, acquisition, and applications. IEEE Trans Neural Networks Learn Syst 33(2):494–514MathSciNetCrossRef
20.
Joshi M, Chen D, Liu Y, Weld DS, Zettlemoyer L, Levy O (2020) Spanbert: improving pre-training by representing and predicting spans. Trans Assoc Comput Linguist 8:64–77CrossRef
21.
Khashabi D, Khot T, Sabharwal A, Tafjord O, Clark P, Hajishirzi H (2020) Unifiedqa: Crossing format boundaries with a single qa system. arXiv preprint arXiv:​2005.​00700
22.
23.
Lan Z, Chen M, Goodman S, Gimpel K, Sharma P, Soricut R (2019) Albert: A lite bert for self-supervised learning of language representations. arXiv preprint arXiv:​1909.​11942
24.
Li L, Gan Z, Cheng Y, Liu J (2019a) Relation-aware graph attention network for visual question answering. In: Proceedings of the IEEE/cvf international conference on computer vision, pp 10313–10322
25.
Li S, Chen J, Yu D (2019b) Teaching pretrained models with commonsense reasoning: a preliminary kb-based approach. arXiv preprint arXiv:​1909.​09743
26.
Lin BY, Chen X, Chen J, Ren X (2019) Kagnet: Knowledge-aware graph networks for commonsense reasoning. In: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), pp 2822–2832
27.
Liu Y, Ott M, Goyal N, Du J, Joshi M, Chen D, Levy O, Lewis M, Zettlemoyer L, Stoyanov V (2019) Roberta: a robustly optimized bert pretraining approach. arXiv preprint arXiv:​1907.​11692
28.
Liu Y, Wan Y, He L, Peng H, Yu PS (2021) Kg-bart: knowledge graph augmented bart for generative commonsense reasoning. Proceedings of the AAAI Conference on Artificial Intelligence 35:6418–6425CrossRef
29.
30.
Lukovnikov D, Fischer A, Lehmann J, Auer S (2017) Neural network-based question answering over knowledge graphs on word and character level. In: Proceedings of the 26th international conference on World Wide Web, pp 1211–1220
31.
Lv S, Guo D, Xu J, Tang D, Duan N, Gong M, Shou L, Jiang D, Cao G, Hu S (2020) Graph-based reasoning over heterogeneous external knowledge for commonsense question answering. Proceedings of the AAAI Conference on Artificial Intelligence 34:8449–8456CrossRef
32.
Ma K, Francis J, Lu Q, Nyberg E, Oltramari A (2019) Towards generalizable neuro-symbolic systems for commonsense question answering. arXiv preprint arXiv:​1910.​14087
33.
Ma K, Ilievski F, Francis J, Bisk Y, Nyberg E, Oltramari A (2021) Knowledge-driven data construction for zero-shot evaluation in commonsense question answering. Proceedings of the AAAI Conference on Artificial Intelligence 35:13507–13515CrossRef
34.
Mihaylov T, Clark P, Khot T, Sabharwal A (2018) Can a suit of armor conduct electricity? a new dataset for open book question answering. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp 2381–2391
35.
Mikolov T, Chen K, Corrado G, Dean J (2013) Efficient estimation of word representations in vector space. arXiv preprint arXiv:​1301.​3781
36.
Mishra BD, Tandon N, Clark P (2017) Domain-targeted, high precision knowledge extraction. Trans Assoc Comput Linguist 5:233–246CrossRef
37.
Mitra A, Banerjee P, Pal KK, Mishra S, Baral C (2019) How additional knowledge can improve natural language commonsense question answering? arXiv preprint arXiv:​1909.​08855
38.
Na SH, Lee JH (2020) Jbnu at semeval-2020 task 4: Bert and unilm for commonsense validation and explanation. In: Proceedings of the fourteenth workshop on semantic evaluation, pp 527–534
39.
Nickel M, Murphy K, Tresp V, Gabrilovich E (2015) A review of relational machine learning for knowledge graphs. Proc IEEE 104(1):11–33CrossRef
40.
Nickel M, Rosasco L, Poggio T (2016) Holographic embeddings of knowledge graphs. In: Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, pp 1955–1961
41.
Peters M, Neumann M, Iyyer M, Gardner M, Clark C, Lee K, Zettlemoyer L (2018) Deep contextualized word representations. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, vol 1, pp 2227–2237
42.
Raffel C, Shazeer N, Roberts A, Lee K, Narang S, Matena M, Zhou Y, Li W, Liu PJ (2019) Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv preprint arXiv:​1910.​10683
43.
Romero J, Razniewski S, Pal K, Z Pan J, Sakhadeo A, Weikum G (2019) Commonsense properties from query logs and question answering forums. In: Proceedings of the 28th ACM International Conference on Information and Knowledge Management, pp 1411–1420
44.
Saeedi S, Panahi A, Saeedi S, Fong AC (2020) Cs-nlp team at semeval-2020 task 4: evaluation of state-of-the-art nlp deep learning architectures on commonsense reasoning task. arXiv preprint arXiv:​2006.​01205
45.
Saha A, Pahuja V, Khapra MM, Sankaranarayanan K, Chandar S (2018) Complex sequential question answering: towards learning to converse over linked question answer pairs with a knowledge graph. In: Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence and Thirtieth Innovative Applications of Artificial Intelligence Conference and Eighth AAAI Symposium on Educational Advances in Artificial Intelligence, pp 705–713
46.
Santoro A, Raposo D, Barrett DG, Malinowski M, Pascanu R, Battaglia P, Lillicrap T (2017) A simple neural network module for relational reasoning. In: Advances in neural information processing systems, pp 4967–4976
47.
Sap M, Le Bras R, Allaway E, Bhagavatula C, Lourie N, Rashkin H, Roof B, Smith NA, Choi Y (2019) Atomic: An atlas of machine commonsense for if-then reasoning. Proceedings of the AAAI Conference on Artificial Intelligence 33:3027–3035CrossRef
48.
Speer R, Chin J, Havasi C (2017) Conceptnet 5.5: an open multilingual graph of general knowledge. In: Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, pp 4444–4451
49.
Schlichtkrull M, Kipf TN, Bloem P, Van Den Berg R, Titov I, Welling M (2018) Modeling relational data with graph convolutional networks. In: European semantic web conference, Springer, pp 593–607
50.
Speer R, Chin J, Havasi C (2017) Conceptnet 5.5: an open multilingual graph of general knowledge. In: Proceedings of the AAAI conference on artificial intelligence, vol 31
51.
Strathearn C, Gkatzia D (2021) Chefbot: A novel framework for the generation of commonsense-enhanced responses for task-based dialogue systems. In: Proceedings of the 14th International Conference on Natural Language Generation, pp 46–47
52.
Sun H, Dhingra B, Zaheer M, Mazaitis K, Salakhutdinov R, Cohen W (2018) Open domain question answering using early fusion of knowledge bases and text. In: Proceedings of the 2018 conference on empirical methods in natural language processing, pp 4231–4242
53.
Talmor A, Herzig J, Lourie N, Berant J (2019) Commonsenseqa: A question answering challenge targeting commonsense knowledge. In: Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, vol 1, pp 4149–4158
54.
Tandon N, De Melo G, Weikum G (2017) Webchild 2.0: fine-grained commonsense knowledge distillation. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics-System Demonstration, pp 115–120
55.
Wan G, Pan S, Gong C, Zhou C, Haffari G (2021) Reasoning like human: Hierar-chical reinforcement learning for knowledge graph reasoning. In: Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence, pp 1926–1932
56.
Wang C, Liang S, Jin Y, Wang Y, Zhu X, Zhang Y (2020a) Semeval-2020 task 4: commonsense validation and explanation. arXiv preprint arXiv:​2007.​00236
57.
Wang G, Hou X, Yang D, McKeown K, Huang J (2021) Semantic categorization of social knowledge for commonsense question answering. arXiv preprint arXiv:​2109.​05168
58.
Wang H, Tang X, Lai S, Leung KS, Zhu J, Fung GPC, Wong KF (2020b) Cuhk at semeval-2020 task 4: Commonsense explanation, reasoning and prediction with multi-task learning. arXiv preprint arXiv:​2006.​09161
59.
Wang P, Peng N, Szekely P, Ren X (2020c) Connecting the dots: a knowledgeable path generator for commonsense question answering. arXiv preprint arXiv:​2005.​00691
60.
Wang X, Kapanipathi P, Musa R, Yu M, Talamadupula K, Abdelaziz I, Chang M, Fokoue A, Makni B, Mattei N et al (2019) Improving natural language inference using external knowledge in the science questions domain. Proceedings of the AAAI Conference on Artificial Intelligence 33:7208–7215CrossRef
61.
Wu S, Li Y, Zhang D, Zhou Y, Wu Z (2020) Topicka: generating commonsense knowledge-aware dialogue responses towards the recommended topic fact. In: Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence, pp 3766–3772
62.
Yan J, Raman M, Zhang T, Rossi R, Zhao H, Kim S, Lipka N, Ren X (2020) Learning contextualized knowledge structures for commonsense reasoning. arXiv preprint arXiv:​2010.​12873
63.
Yang Z, Dai Z, Yang Y, Carbonell J, Salakhutdinov RR, Le QV (2019) Xlnet: generalized autoregressive pretraining for language understanding. In: Proceedings of the 33rd International Conference on Neural Information Processing Systems, pp 5753–5763
64.
Yasunaga M, Ren H, Bosselut A, Liang P, Leskovec J (2021) Qa-gnn: Reasoning with language models and knowledge graphs for question answering. arXiv preprint arXiv:​2104.​06378
65.
Ye ZX, Chen Q, Wang W, Ling ZH (2019) Align, mask and select: a simple method for incorporating commonsense knowledge into language representation models. arXiv preprint arXiv:​1908.​06725
66.
Young T, Cambria E, Chaturvedi I, Zhou H, Biswas S, Huang M (2018) Augmenting end-to-end dialogue systems with commonsense knowledge. In: Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence and Thirtieth Innovative Applications of Artificial Intelligence Conference and Eighth AAAI Symposium on Educational Advances in Artificial Intelligence, pp 4970–4977
67.
Zhan X, Huang Y, Dong X, Cao Q, Liang X (2022) Pathreasoner: Explainable reasoning paths for commonsense question answering. Knowl Based Syst 235:107612CrossRef
68.
Zhang Y, Dai H, Kozareva Z, Smola A, Song L (2018) Variational reasoning for question answering with knowledge graph. In: Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence and Thirtieth Innovative Applications of Artificial Intelligence Conference and Eighth AAAI Symposium on Educational Advances in Artificial Intelligence, pp 6069–6076
69.
Zhu C, Cheng Y, Gan Z, Sun S, Goldstein T, Liu J (2019) Freelb: Enhanced adversarial training for natural language understanding. arXiv preprint arXiv:​1909.​11764
70.
Zhan X, Huang Y, Dong X, Cao Q, Liang X (2022) Pathreasoner: Explainable reasoning paths for commonsense question answering. Knowl Based Syst 235:107612CrossRef
71.
Huang Y, Fang M, Zhan X, Cao Q, Liang X, Lin L (2021) Rem-net: Recursive erasure memory network for commonsense evidence refinement. Proceedings of the AAAI Conference on Artificial Intelligence 35:6375–6383CrossRef
Metadaten
Titel
Implicit Relation Inference with Deep Path Extraction for Commonsense Question Answering
verfasst von
Peng Yang
Zijian Liu
Bing Li
Penghui Zhang
Publikationsdatum
11.05.2022
Verlag
Springer US
Erschienen in
Neural Processing Letters / Ausgabe 6/2022
Print ISSN: 1370-4621
Elektronische ISSN: 1573-773X
DOI
https://doi.org/10.1007/s11063-022-10831-8

Weitere Artikel der Ausgabe 6/2022

Neural Processing Letters 6/2022 Zur Ausgabe

OriginalPaper

FuseNet: 3D Object Detection Network with Fused Information for Lidar Point Clouds

OriginalPaper

Color Image Enhancement: A Metaheuristic Chimp Optimization Algorithm

OriginalPaper

Security Event-Triggered Filtering for Delayed Neural Networks Under Denial-of-Service Attack and Randomly Occurring Deception Attacks

OriginalPaper

A dark image enhancement method based on multiscale features and dilated residual networks

OriginalPaper

A State Recognition Method of Isolation Switch in Traction Substation Based on Key Components Detection and Geometric Ranging

OriginalPaper

CSA-Net: Deep Cross-Complementary Self Attention and Modality-Specific Preservation for Saliency Detection

Neuer Inhalt

    Bildnachweise