nach oben

Erschienen in:

01.12.2023

Heterogeneous graph attention networks for passage retrieval

verfasst von: Lucas Albarede, Philippe Mulhem, Lorraine Goeuriot, Sylvain Marié, Claude Le Pape-Gardeux, Trinidad Chardin-Segui

Erschienen in: Discover Computing | Ausgabe 1-2/2023

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

This paper presents an exploration of the usage of Heterogeneous Graph Attention Networks, or HGATs, for the task of Passage Retrieval. More precisely, we study how these models perform to alleviate the problem of passage contextualization, that is incorporating information about the context of a passage (its containing document, neighbouring passages, etc.) in its relevance estimation. We first propose several configurations to compute contextualized passage representations, including a document graph representation composed of contextualizing signals and judiciously modified HGAT architectures. We then present how we integrate these configurations in a neural passage ranking model. We evaluate our approach on a Passage Retrieval task on patent documents: CLEF-IP2013, as these documents possess several different contextualizing signals fully exploited in our models. Our results show that some HGAT architecture modifications allow for a better context representation leading to improved performances and stability.

Vorheriger Artikel Constructing and meta-evaluating state-aware evaluation metrics for interactive search systems

Nächster Artikel Privacy-aware document retrieval with two-level inverted indexing

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Nur mit Berechtigung zugänglich

http://homepages.inf.ed.ac.uk/wmagdy/PRES.html.

http://www.ifs.tuwien.ac.at/clef-ip/index.html

Albarede, L., Mulhem, P., Goeuriot, L., Le Pape-Gardeux, C., Marie, S. and Chardin-Segui, T. (2021). Passage retrieval in context: Experiments on patents. Proceedings of CORIA’21. Proceedings of coria’21. Grenoble, France. https://hal.archives-ouvertes.fr/hal-03230421

Albarede, L., Mulhem, P., Goeuriot, L., Le Pape-Gardeux, C., Marié, S. and Chardin-Segui, T. (2022). Passage Retrieval on Structured Documents using Graph Attention Networks. Proceedings of ECIR 2022. Proceedings of ecir 2022. Stavanger, Norway. https://hal.archives-ouvertes.fr/hal-03626054

Andersson, L., Lupu, M., Palotti, J.a., Hanbury, A. & Rauber, A. (2016). When is the time ripe for natural language processing for patent passage retrieval? Proceedings of the 25th ACM International on Conference on Information and Knowledge Management Proceedings of the 25th acm international on conference on information and knowledge management (p. 1453-1462). New York, NY, USAAssociation for Computing Machinery. https://doi.org/10.1145/2983323.2983858

Arnold, S., van Aken, B., Grundmann, P., Gers, F.A. and Löser, A. (2020). Learning contextualized document representations for healthcare answer retrieval. CoRR abs/2002.00835) arXiv.org/abs/2002.00835

Arvola, P., Junkkari, M. and Kekäläinen, J. (2005). Generalized contextualization method for xml information retrieval. Proceedings of the 14th ACM International Conference on Information and Knowledge Management Proceedings of the 14th acm international conference on information and knowledge management (p. 20-27). New York, NY, USAAssociation for Computing Machinery. https://doi.org/10.1145/1099554.1099561

Arvola, P., Kekäläinen, J. & Junkkari, M. (2008). The effect of contextualization at different granularity levels in content-oriented xml retrieval. Proceedings of the 17th ACM Conference on Information and Knowledge Management Proceedings of the 17th acm conference on information and knowledge management (p. 1491-1492). New York, NY, USAAssociation for Computing Machinery. https://doi.org/10.1145/1458082.1458350

Bahdanau, D., Cho, K. & Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. arXiv. arxiv:1409.0473

Beigbeder, M. (2010). Focused retrieval with proximity scoring. Proceedings of the 2010 ACM Symposium on Applied Computing Proceedings of the 2010 acm symposium on applied computing (p. 1755-1759). New York, NY, USAAssociation for Computing Machinery. https://doi.org/10.1145/1774088.1774462

Bendersky, M. & Kurland, O. (2008). Utilizing passage-based language models for document retrieval. Proceedings of the IR Research, 30th European Conference on Advances in Information Retrieval Proceedings of the ir research, 30th european conference on advances in information retrieval (p. 162-174). Berlin, HeidelbergSpringer-Verlag.

Callan, J.P. (1994). Passage-level evidence in document retrieval. Proceedings of the 17th annual international acm sigir conference on research and development in information retrieval (p. 302-310). Berlin, HeidelbergSpringer-Verlag.

Carmel, D., Shtok, A. & Kurland, O. (2013). Position-based contextualization for passage retrieval. In: Proceedings of the 22nd acm international conference on information & knowledge management (p. 1241-1244). New York, NY, USAAssociation for Computing Machinery. https://doi.org/10.1145/2505515.2507865

Chen, L., Li, J., Gong, Z., Zhang, M. & Zhou, G. (2022). One type context is not enough: Global context-aware neural machine translation. ACM Transactions on Asian and Low-Resource Language Information Processing. https://doi.org/10.1145/3526215 (Just Accepted)

Cohen, D., Jordan, S.M. & Croft, W.B. (2019) Learning a better negative sampling policy with deep neural networks for search. Proceedings of the 2019 acm sigir international conference on theory of information retrieval (p. 19-26). New York, NY, USAAssociation for Computing Machinery. https://doi.org/10.1145/3341981.3344220

Cui, H., Lu, J., Ge, Y. & Yang, C. (2022) How can graph neural networks help document retrieval: A case study on cord19 with concept map generation. arXiv. arxiv:2201.04672

Fernández, R., Losada, D. & Azzopardi, L. (2011). Extending the language modeling framework for sentence retrieval to include local context. Inf. Retr. 14, 355-389. https://doi.org/10.1007/s10791-010-9146-4

Gobeill, J. & Ruch, P. (2012). Bitem site report for the claims to passage task in CLEF-IP 2012. P. Forner, J. Karlgren and C. Womser-Hacker (Eds.), CLEF 2012 Evaluation Labs and Workshop, Online Working Notes, Rome, Italy, September 17-20, 2012 CLEF 2012 evaluation labs and workshop, online working notes, rome, italy, september 17-20, 2012 (Vol. 1178). CEUR-WS.org. http://ceur-ws.org/Vol-1178/CLEF2012wn-CLEFIP-GobeillEt2012.pdf

Hao, X., Zhou, Y., Wu, D., Zhang, W., Li, B. & Wang, W. (2021). Multi-feature graph attention network for cross-modal video-text retrieval. Proceedings of the 2021 international conference on multimedia retrieval (p. 135-143). New York, NY, USAAssociation for Computing Machinery. https://doi.org/10.1145/3460426.3463608

Hofstätter, S., Mitra, B., Zamani, H., Craswell, N. and Hanbury, A. (2021). Intra-document cascading: Learning to select passages for neural document ranking. arXiv. arxiv:2105.09816

Hong, W., Zhang, Z., Wang, J. and Zhao, H. (2022). Sentence-aware contrastive learning for open-domain passage retrieval. Proceedings of the 60th annual meeting of the association for computational linguistics (volume 1: Long papers) (p.S 1062–1074). Dublin, IrelandAssociation for Computational Linguistics. https://aclanthology.org/2022.acl-long.76

Kaszkiel, M., Zobel, J. & Sacks-Davis, R. (1999). Efficient passage ranking for document databases. ACM Transactions on Information Systems (TOIS) 17(4), 406-439. https://doi.org/10.1145/326440.326445

Kekäläinen, J., Arvola, P. & Junkkari, M. (2018). Contextualization in structured text retrieval. In L. Liu & M.T. Ozsu (Eds.),Encyclopedia of Database Systems Encyclopedia of database systems (p.S 611–613). New York, NYSpringer New York. https://doi.org/10.1007/978-1-4614-8265-9_81

Khattab, O. & Zaharia, M. (2020). Colbert: Efficient and effective passage search via contextualized late interaction over BERT. CoRR, abs/2004.12832) arxiv:2004.12832

Kingma, D.P. & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv. arxiv:1412.6980

Krikon, E., Kurland, O., & Bendersky, M. (2011). Utilizing inter-passage and inter-document similarities for reranking search results. ACM Transactions on Information Systems, 29(1), 1–28. https://doi.org/10.1145/1877766.1877769CrossRef

Li, X., de Rijke, M., Liu, Y., Mao, J., Ma, W., Zhang, M. and Ma, S. (2020). Learning better representations for neural information retrieval with graph information. Proceedings of the 29th acm international conference on information & knowledge management (p. 795-804). New York, NY, USA. Association for Computing Machinery. https://doi.org/10.1145/3340531.3411957

Liu, J., Liu, J., Yang, Y., Wang, J., Wu, W., Zhao, D. and Yan, R. (2022). Gnn-encoder: Learning a dual-encoder architecture via graph neural networks for passage retrieval. arXiv. arxiv:2204.08241

Liu, Y., Hashimoto, K., Zhou, Y., Yavuz, S., Xiong, C. and Yu, P.S. (2021a). Dense hierarchical retrieval for open-domain question answering. arXiv. arxiv:2110.15439

Liu, Y., Hashimoto, K., Zhou, Y., Yavuz, S., Xiong, C. and Yu, P.S. (2021b). Dense hierarchical retrieval for open-domain question answering. arXiv. arxiv:2110.15439

Lu, J., Ábrego, G.H., Ma, J., Ni, J. and Yang, Y. (2020). Neural passage retrieval with improved negative contrast. CoRR, abs/2010.12523 arxiv:2010.12523

Lu, J., Hernandez Abrego, G., Ma, J., Ni, J. and Yang, Y. (2021). Multi-stage training with improved negative contrast for neural passage retrieval. Proceedings of the 2021 conference on empirical methods in natural language processing (p.S 6091–6103). Online and Punta Cana, Dominican Republic Association for Computational Linguistics. https://aclanthology.org/2021.emnlp-main.492 https://doi.org/10.18653/v1/2021.emnlp-main.492

Macdonald, C. & Tonellotto, N. (2020). Declarative experimentation in information retrieval using PyTerrier. Proceedings of the 2020 ACM SIGIR on international conference on theory of information retrieval. ACM. https://doi.org/10.1145/3409256.3409829

Macdonald, C., Tonellotto, N. and Ounis, I. (2021). On single and multiple representations in dense passage retrieval. CoRR, abs/2108.06279) arxiv:2108.06279

Mahdabi, P., Gerani, S., Huang, J.X. and Crestani, F. (2013). Leveraging conceptual lexicon: Query disambiguation using proximity information for patent retrieval. Proceedings of the 36th international acm sigir conference on research and development in information retrieval (p. 113-122). New York, NY, USAAssociation for Computing Machinery. https://doi.org/10.1145/2484028.2484056

Mahdabi, P., Keikha, M., Gerani, S., Landoni, M. and Crestani, F. (2011). Building queries for prior-art search. A. Hanbury, A. Rauber and A.P. de Vries (Eds.), Multidisciplinary information retrieval (p.S 3–15). Berlin, HeidelbergSpringer Berlin Heidelberg.

Mass, Y. & Mandelbrod, M. (2005). Component ranking and automatic query refinement for xml retrieval. N. Fuhr, M. Lalmas and S. Malik (Eds.), Advances in xml information retrieval (p.S 73–84). Berlin, HeidelbergSpringer Berlin Heidelberg.

Murdock, V. & Croft, W.B. (2005). A translation model for sentence retrieval. Proceedings of human language technology conference and conference on empirical methods in natural language processing (p.S 684–691). Vancouver, British Columbia, CanadaAssociation for Computational Linguistics. https://www.aclweb.org/anthology/H05-1086

Nguyen, T., Rosenberg, M., Song, X., Gao, J., Tiwary, S., Majumder, R. and Deng, L. (2016). MS MARCO: A human generated machine reading comprehension dataset. CoRR, abs/1611.09268) http://arxiv.org/abs/1611.09268

Nogueira, R.F. & Cho, K. (2019). Passage re-ranking with BERT. CoRR, abs/1901.04085) arxiv:1901.04085

Norozi, M.A. & Arvola, P. (2013). Kinship contextualization: Utilizing the preceding and following structural elements. Proceedings of the 36th international acm sigir conference on research and development in information retrieval (p. 837-840). New York, NY, USAAssociation for Computing Machinery. https://doi.org/10.1145/2484028.2484111

Norozi, M.A., Arvola, P. and de Vries, A.P. (2012). Contextualization using hyperlinks and internal hierarchical structure of wikipedia documents. Proceedings of the 21st acm international conference on information and knowledge management (p. 734-743). New York, NY, USAAssociation for Computing Machinery. https://doi.org/10.1145/2396761.2396855

Norozi, M.A., de Vries, A. and Arvola, P. (2012). Contextualization from the Bibliographic Structure. Proceeding of the ecir 2012 workshop on task-based and aggregated search (tbas2012), page 9.

Ogilvie, P. & Callan, J. (2005). Hierarchical language models for xml component retrieval. N. Fuhr, M. Lalmas and S. Malik (Eds.), Advances in xml information retrieval (p.S 224–237). Berlin, HeidelbergSpringer Berlin Heidelberg.

Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G. and Chintala, S. (2019). Pytorch: An imperative style, high-performance deep learning library. H. Wallach, H. Larochelle, A. Beygelzimer, F. d’ Alché-Buc, E. Fox and R. Garnett (Eds.), Advances in neural information processing systems 32 (p.S 8024–8035). Curran Associates, Inc. http://papers.neurips.cc/paper/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.pdf

Piroi, F., Lupu, M. and Hanbury, A. (2013). Overview of clef-ip 2013 lab. P. Forner, H. Müller, R. Paredes, P. Rosso and B. Stein (Eds.), Information access evaluation. multilinguality, multimodality, and visualization (p.S 232–249). Berlin, HeidelbergSpringer Berlin Heidelberg.

Qi, Y., Zhang, J., Liu, Y., Xu, W. and Guo, J. (2020). Cgtr: Convolution graph topology representation for document ranking. Proceedings of the 29th ACM International Conference on Information & Knowledge Management (p. 2173-2176). New York, NY, USAAssociation for Computing Machinery. https://doi.org/10.1145/3340531.3412073

Robertson, S., Walker, S., Jones, S., Hancock-Beaulieu, M.M. and Gatford, M. (1995). Okapi at trec-3. Overview of the third text retrieval conference (trec-3) (Overview of the Third Text REtrieval Conference (TREC-3) ed., p. 109-126). Gaithersburg, MD: NIST. https://www.microsoft.com/en-us/research/publication/okapi-at-trec-3/

Santhanam, K., Khattab, O., Potts, C. and Zaharia, M.A. (2022). Plaid: An efficient engine for late interaction retrieval. ArXiv:abs/2205.09707

Sheetrit, E., Shtok, A. and Kurland, O. (2019). A passage-based approach to learning to rank documents. ArXiv:abs/1906.02083

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N. and Polosukhin, I. (2017). Attention is all you need. arXiv. arxiv:1706.03762

Veličković, P., Cucurull, G., Casanova, A., Romero, A., Liò, P. and Bengio, Y. (2017). Graph attention networks. arXiv. arxiv:1710.10903

Wang, X., Ji, H., Shi, C., Wang, B., Cui, P., Yu, P.S. and Ye, Y. (2019). Heterogeneous graph attention network. CoRR. arXiv:1903.07293

Wu, N., Liang, Y., Ren, H., Shou, L., Duan, N., Gong, M. and Jiang, D. (2022). Unsupervised context aware sentence representation pretraining for multi-lingual dense retrieval.

Xie, Q., Huang, J., Du, P., Peng, M. and Nie, J.-Y. (2021). Graph topic neural network for document representation. Proceedings of the web conference 2021 (p. 3055-3065). New York, NY, USA. Association for Computing Machinery. https://doi.org/10.1145/3442381.3450045

Xu, P., Chen, X., Ma, X., Huang, Z. and Xiang, B. (2021). Contrastive document representation learning with graph attention networks. arXiv. arxiv:2110.10778

Xue, X. & Croft, W.B. (2009). Automatic query generation for patent search. Proceedings of the 18th acm conference on information and knowledge management (p. 2037-2040). New York, NY, USAAssociation for Computing Machinery. https://doi.org/10.1145/1645953.1646295

Zhang, T., Liu, B., Niu, D., Lai, K. and Xu, Y. (2018). Multiresolution graph attention networks for relevance matching. Proceedings of the 27th ACM International Conference on Information and Knowledge Management) https://doi.org/10.1145/3269206.3271806

Zhang, Y., Liu, C., Luo, A., Xue, H., Shan, X., Luo, Y. and Wang, H. (2021). Mira:leveraging multi-intention co-click information in web-scale document retrieval using deep neural networks. Proceedings of the Web Conference 2021 Proceedings of the web conference 2021 (p. 227-238). New York, NY, USAAssociation for Computing Machinery. https://doi.org/10.1145/3442381.3449865

Zhao, S., Su, C., Sboner, A. & Wang, F. (2019). GRAPHENE. Proceedings of the 28th ACM International Conference on Information and Knowledge Management. Proceedings of the 28th ACM international conference on information and knowledge management. ACM. https://doi.org/10.1145/3357384.3358038

Titel: Heterogeneous graph attention networks for passage retrieval
verfasst von: Lucas Albarede
Philippe Mulhem
Lorraine Goeuriot
Sylvain Marié
Claude Le Pape-Gardeux
Trinidad Chardin-Segui
Publikationsdatum: 01.12.2023
Verlag: Springer Netherlands
Erschienen in: Discover Computing / Ausgabe 1-2/2023
Print ISSN: 2948-2984
Elektronische ISSN: 2948-2992
DOI: https://doi.org/10.1007/s10791-023-09424-3

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Weitere Artikel der Ausgabe 1-2/2023

MuMUR: Multilingual Multimodal Universal Retrieval

Tashaphyne0.4: a new arabic light stemmer based on rhyzome modeling approach

FarsNewsQA: a deep learning-based question answering system for the Persian news articles

Shop by image: characterizing visual search in e-commerce

Constructing and meta-evaluating state-aware evaluation metrics for interactive search systems

An in-depth analysis of passage-level label transfer for contextual document ranking

Premium Partner