Top

Published in:

2020 | OriginalPaper | Chapter

Question Answering When Knowledge Bases are Incomplete

Authors : Camille Pradel, Damien Sileo, Álvaro Rodrigo, Anselmo Peñas, Eneko Agirre

Published in: Experimental IR Meets Multilinguality, Multimodality, and Interaction

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

While systems for question answering over knowledge bases (KB) continue to progress, real world usage requires systems that are robust to incomplete KBs. Dependence on the closed world assumption is highly problematic, as in many practical cases the information is constantly evolving and KBs cannot keep up. In this paper we formalize a typology of missing information in knowledge bases, and present a dataset based on the Spider KB question answering dataset, where we deliberately remove information from several knowledge bases, in this case implemented as relational databases (The dataset and the code to reproduce experiments are available at https://github.com/camillepradel/IDK.). Our dataset, called IDK (Incomplete Data in Knowledge base question answering), allows to perform studies on how to detect and recover from such cases. The analysis shows that simple baselines fail to detect most of the unanswerable questions.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

previous chapter Query or Document Translation for Academic Search – What’s the Real Difference?

next chapter 2AIRTC: The Amharic Adhoc Information Retrieval Test Collection

https://www.w3.org/standards/semanticweb/.

None type corresponds to RDF plain literals: https://www.w3.org/TR/rdf-concepts/#dfn-plain-literal.

Plus domain and range properties, and labels language tags we did not include in our definition for the sake of simplicity.

We used Magnitude [12] in order to query the embeddings in a way that is robust to minor morphological word differences.

Artzi, Y., Zettlemoyer, L.: Weakly supervised learning of semantic parsers for mapping instructions to actions. Trans. Assoc. Comput. Linguist. 1, 49–62 (2013)CrossRef

Berant, J., Chou, A., Frostig, R., Liang, P.: Semantic parsing on Freebase from question-answer pairs. In: Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, Seattle, Washington, USA, pp. 1533–1544. Association for Computational Linguistics, October 2013. https://www.aclweb.org/anthology/D13-1160

Bogin, B., Gardner, M., Berant, J.: Representing schema structure with graph neural networks for text-to-SQL parsing. In: ACL (2019)

Chebotko, A., Lu, S., Fotouhi, F.: Semantics preserving SPARQL-to-SQL translation. Data Knowl. Eng. 68, 973–1000 (2009)CrossRef

Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, (Long and Short Papers), Minneapolis, Minnesota, vol. 1, pp. 4171–4186. Association for Computational Linguistics, June 2019. https://doi.org/10.18653/v1/N19-1423. https://www.aclweb.org/anthology/N19-1423

Guo, J., et al.: Towards complex text-to-SQL in cross-domain database with intermediate representation. arXiv preprint arXiv:1905.08205 (2019). Version 2

Hixon, B., Clark, P., Hajishirzi, H.: Learning knowledge graphs for question answering through conversational dialog. In: Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Denver, Colorado, pp. 851–861. Association for Computational Linguistics, May–June 2015. https://doi.org/10.3115/v1/N15-1086. https://www.aclweb.org/anthology/N15-1086

Joshi, M., Choi, E., Weld, D.S., Zettlemoyer, L.: Triviaqa: a large scale distantly supervised challenge dataset for reading comprehension. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, Vancouver, Canada. Association for Computational Linguistics, July 2017

Lopez, V., Unger, C., Cimiano, P., Motta, E.: Evaluating question answering over linked data. J. Web Semant. (2013). https://doi.org/10.1016/j.websem.2013.05.006

10.

Mikolov, T., Grave, E., Bojanowski, P., Puhrsch, C., Joulin, A.: Advances in pre-training distributed word representations. In: Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Resources Association (ELRA), May 2018. https://www.aclweb.org/anthology/L18-1008

11.

Pasupat, P., Liang, P.: Compositional semantic parsing on semi-structured tables. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), Beijing, China, pp. 1470–1480. Association for Computational Linguistics, July 2015. https://doi.org/10.3115/v1/P15-1142. https://www.aclweb.org/anthology/P15-1142

12.

Patel, A., Sands, A., Callison-Burch, C., Apidianaki, M.: Magnitude: a fast, efficient universal vector embedding utility package. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, Brussels, Belgium, pp. 120–126. Association for Computational Linguistics, November 2018. https://doi.org/10.18653/v1/D18-2021. https://www.aclweb.org/anthology/D18-2021

13.

Peñas, A., Veron, M., Pradel, C., Otegi, A., Echegoyen, G., Rodrigo, A.: Continuous learning for question answering. In: Proceedings of the 10th International Workshop on Spoken Dialog Systems (IWSDS 2019) - DSLL Special Session (2019)

14.

Price, P.J.: Evaluation of spoken language systems: the ATIS domain. In: Speech and Natural Language: Proceedings of a Workshop Held at Hidden Valley, Pennsylvania, 24–27 June 1990 (1990). https://www.aclweb.org/anthology/H90-1020

15.

Rajpurkar, P., Jia, R., Liang, P.: Know what you don’t know: unanswerable questions for SQuAD. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), Melbourne, Australia, pp. 784–789. Association for Computational Linguistics, July 2018. https://doi.org/10.18653/v1/P18-2124. https://www.aclweb.org/anthology/P18-2124

16.

Rajpurkar, P., Zhang, J., Lopyrev, K., Liang, P.: SQuAD: 100,000+ questions for machine comprehension of text. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin, Texas, pp. 2383–2392. Association for Computational Linguistics, November 2016. https://doi.org/10.18653/v1/D16-1264. https://www.aclweb.org/anthology/D16-1264

17.

Soussi, N., Bahaj, M.: Semantics preserving SQL-to-SPARQL query translation for nested right and left outer join. J. Appl. Res. Technol. 15(5), 504–512 (2017)CrossRef

18.

Yu, T., et al.: CoSQL: a conversational text-to-SQL challenge towards cross-domain natural language interfaces to databases. In: Proceedings of EMNLP 2019 (2019)

19.

Yu, T., et al.: Spider: a large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-SQL task. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, pp. 3911–3921. Association for Computational Linguistics, October–November 2018. https://doi.org/10.18653/v1/D18-1425. https://www.aclweb.org/anthology/D18-1425

20.

Yu, T., et al.: SParC: cross-domain semantic parsing in context. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy. Association for Computational Linguistics (2019)

21.

Zelle, J.M., Mooney, R.J.: Learning to parse database queries using inductive logic programming. In: Proceedings of the Thirteenth National Conference on Artificial Intelligence, AAAI 1996, vol. 2, pp. 1050–1055. AAAI Press (1996). http://dl.acm.org/citation.cfm?id=1864519.1864543

22.

Zhong, V., Xiong, C., Socher, R.: Seq2SQL: generating structured queries from natural language using reinforcement learning. arXiv preprint arXiv:1709.00103 (2017)

Title: Question Answering When Knowledge Bases are Incomplete
Authors: Camille Pradel
Damien Sileo
Álvaro Rodrigo
Anselmo Peñas
Eneko Agirre
Publisher: Springer International Publishing
Book: Experimental IR Meets Multilinguality, Multimodality, and Interaction
Print ISBN: 978-3-030-58218-0

Electronic ISBN: 978-3-030-58219-7

Copyright Year: 2020
DOI: https://doi.org/10.1007/978-3-030-58219-7_4

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partner