Top

Cognitive Computation

27-07-2024 | Research

Multi-View Cooperative Learning with Invariant Rationale for Document-Level Relation Extraction

Authors: Rui Lin, Jing Fan, Yinglong He, Yehui Yang, Jia Li, Cunhan Guo

Published in: Cognitive Computation

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Document-level relation extraction (RE) is a complex and significant natural language processing task, as the massive entity pairs exist in the document and are across sentences in reality. However, the existing relation extraction methods (deep learning) often use single-view information (e.g., entity-level or sentence-level) to learn the relational information but ignore the multi-view information, and the explanations of deep learning are difficult to be reflected, although it achieves good results. To extract high-quality relational information from the document and improve the explanations of the model, we propose a multi-view cooperative learning with invariant rationale (MCLIR) framework. Firstly, we design the multi-view cooperative learning to find latent relational information from the various views. Secondly, we utilize invariant rationale to encourage the model to focus on crucial information, which can empower the performance and explanations of the model. We conduct the experiment on two public datasets, and the results of the experiment demonstrate the effectiveness of MCLIR.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

https://cloud.tsinghua.edu.cn/d/99e1c0805eb64736af95/.

https://github.com/huggingface/transformers.

Yao Y, Ye D, Li P, Han X, Lin Y, Liu Z, Liu Z, Huang L, Zhou J, Sun M. DocRED: a large-scale document-level relation extraction dataset. In: Proceedings of ACL, 2019; pp. 764–777. https://www.aclweb.org/anthology/P19-1074/.

Devlin J, Chang M, Lee K, Toutanova K. BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of NAACL-HLT, 2019; pp. 4171–4186. https://www.aclweb.org/anthology/N19-1423/.

Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I. Attention is all you need. In: Proceedings of NIPS, 2017; pp. 6000–6010.

Sennrich R, Haddow B, Birch A. Neural machine translation of rare words with subword units. In: Proceedings of ACL, 2016. https://www.aclweb.org/anthology/P16-1162/.

Wei Y, Wang X, Guan W, Nie L, Lin Z, Chen B. Neural multimodal cooperative learning toward micro-video understanding. IEEE Trans Image Process. 2020;29:1–14. https://doi.org/10.1109/TIP.2019.2923608.MathSciNetCrossRef

Ji G, Liu K, He S, Zhao J. Distant supervision for relation extraction with sentence-level attention and entity descriptions. In: Proceedings of AAAI, 2017; pp. 3060–3066.

Bordes A, Usunier N, García-Durán A, Weston J, Yakhnenko O. Translating embeddings for modeling multi-relational data. In: Advances in Neural Information Processing Systems, 2017; pp. 2787–2795. http://papers.nips.cc/paper/5071-translating-embeddings-for-modeling-multi-relational-data.

Zhang T, Li Z, Shin M, Wang C, Song W, Lui L. Feature extraction method of snowboard starting action using vision sensor image processing. Mob Inf Syst. 2022;2022:2829547–128295479. https://doi.org/10.1155/2022/2829547.CrossRef

Reali P, Lolatto R, Coelli S, Tartaglia G, Bianchi AM. Information retrieval from photoplethysmographic sensors: a comprehensive comparison of practical interpolation and breath-extraction techniques at different sampling rates. Sensors. 2022;22(4):1428. https://doi.org/10.3390/s22041428.CrossRef

10.

Nan G, Guo Z, Sekulic I, Lu W. Reasoning with latent structure refinement for document-level relation extraction. In: Jurafsky, D., Chai, J., Schluter, N., Tetreault, J.R. (eds.) Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, 2020; pp. 1546–1557. https://doi.org/10.18653/v1/2020.acl-main.141.

11.

Tang H, Cao Y, Zhang Z, Cao J, Fang F, Wang S, Yin P. HIN: hierarchical inference network for document-level relation extraction. In: Lauw, H.W., Wong, R.C., Ntoulas, A., Lim, E., Ng, S., Pan, S.J. (eds.) Advances in Knowledge Discovery and Data Mining - 24th Pacific-Asia Conference, PAKDD 2020, Singapore, May 11-14, 2020, Proceedings, Part I. Lecture Notes in Computer Science. 2020;12084:197–209. https://doi.org/10.1007/978-3-030-47426-3_16.

12.

Velickovic P, Cucurull G, Casanova A, Romero A, Liò P, Bengio Y. Graph attention networks. 2017. CoRR abs/1710.10903.

13.

Verga P, Strubell E, McCallum A. Simultaneously self-attending to all mentions for full-abstract biological relation extraction. In: Proceedings of NAACL, 2018; pp. 872–884. https://www.aclweb.org/anthology/N18-1080/.

14.

Sahu SK, Christopoulou F, Miwa M, Ananiadou S. Inter-sentence relation extraction with document-level graph convolutional neural network. In: Proceedings of ACL, 2019; pp. 4309–4316.https://doi.org/10.18653/v1/p19-1423.

15.

Christopoulou F, Miwa M, Ananiadou S. Connecting the dots: document-level neural relation extraction with edge-oriented graphs. In: Proceedings of EMNLP, 2019; pp. 4924–4935. https://doi.org/10.18653/v1/D19-1498.

16.

Guo Z, Zhang Y, Lu W. Attention guided graph convolutional networks for relation extraction. In: Korhonen, A., Traum, D.R., Màrquez, L. (eds.) Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers, 2019; pp. 241–251. https://doi.org/10.18653/v1/p19-1024.

17.

Wang H, Focke C, Sylvester R, Mishra N, Wang W. Fine-tune BERT for DocRED with two-step process. 2019, CoRR abs/1909.11898. arXiv:1909.11898.

18.

Ye D, Lin Y, Du J, Liu Z, Li P, Sun M, Liu Z. Coreferential reasoning learning for language representation. In: Webber, B., Cohn, T., He, Y., Liu, Y. (eds.) Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, 2020; pp. 7170–7186. https://doi.org/10.18653/v1/2020.emnlp-main.582.

19.

Santos CND, Xiang B, Zhou B. Classifying relations by ranking with convolutional neural networks. Comput Sci. 2015;86(86):132–7.

20.

Cho K, Van Merrienboer B, Gulcehre C, Bahdanau D, Bougares F, Schwenk H, Bengio Y. Learning phrase representations using RNN encoder-decoder for statistical machine translation. Comput Sci. 2014.

21.

Liu Y, Wei F, Li S, Ji H, Zhou M, Wang H. A dependency-based neural network for relation classification. In: Proceedings of ACL, pp. 285–290 (2015). http://aclweb.org/anthology/P/P15/P15-2047.pdf.

22.

Zhao D, Wang J, Lin H, Wang X, Yang Z, Zhang Y. Biomedical cross-sentence relation extraction via multihead attention and graph convolutional networks. Appl Soft Comput. 2021;104: 107230. https://doi.org/10.1016/j.asoc.2021.107230.CrossRef

23.

Balaji K, Kumar MS, Yuvaraj N. Multi objective Taguchi-grey relational analysis and krill herd algorithm approaches to investigate the parametric optimization in abrasive water jet drilling of stainless steel. Appl Soft Comput. 2021;102. https://doi.org/10.1016/j.asoc.2020.107075.

24.

Wu H, Ren P, Xu Z. Promoting the physician-patient consensus with a hesitant fuzzy linguistic consensus method based on betweenness relation. Appl Soft Comput. 2022;124: 108979. https://doi.org/10.1016/j.asoc.2022.108979.CrossRef

25.

Chen T, Zhou L, Wang N, Chen X. Joint entity and relation extraction with position-aware attention and relation embedding. Appl Soft Comput. 2022;119: 108604. https://doi.org/10.1016/j.asoc.2022.108604.CrossRef

26.

Miwa M, Bansal M. End-to-end relation extraction using LSTMS on sequences and tree structures. In: Proceedings of ACL, 2016; pp. 1105–1116.

27.

Guo, Z., Zhang, Y., Lu, W.: Attention guided graph convolutional networks for relation extraction. 2019. CoRR abs/1906.07510. arXiv:1906.07510.

28.

Yang B, Mitchell TM. Joint extraction of events and entities within a document context. In: Proceedings of NAACL, 2016; pp. 289–299. https://www.aclweb.org/anthology/N16-1033/.

29.

Swampillai K, Stevenson M. Extracting relations within and across sentences. In: Recent Advances in Natural Language Processing, RANLP 2011, 12-14 September, 2011, Hissar, Bulgaria, 2011; pp. 25–32. https://www.aclweb.org/anthology/R11-1004/.

30.

Jia R, Wong C, Poon H. Document-level n-ary relation extraction with multiscale representation learning. In: Proceedings of NAACL, 2019; pp. 3693–3704. https://www.aclweb.org/anthology/N19-1370/.

31.

Huang Q, Zhu S, Feng Y, Ye Y, Lai Y, Zhao D. Three sentences are all you need: local path enhanced document relation extraction. In: Zong, C., Xia, F., Li, W., Navigli, R. (eds.) Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL/IJCNLP 2021, (Volume 2: Short Papers), Virtual Event, 2021; pp. 998–1004.

32.

Xu W, Chen K, Zhao T. Discriminative reasoning for document-level relation extraction. In: Zong C, Xia F, Li W, Navigli R (eds.) Findings of the Association for Computational Linguistics: ACL/IJCNLP 2021, Online Event, August 1-6, 2021. Findings of ACL, vol. ACL/IJCNLP 2021, 2021; pp. 1653–1663.

33.

Yuan C, Huang H, Feng C, Shi G, Wei X. Document-level relation extraction with entity-selection attention. Inf Sci. 2021;568:163–74. https://doi.org/10.1016/j.ins.2021.04.007.

34.

Wang X, Wei J, Schuurmans D, Le Q, Chi E, Zhou D. Rationale-augmented ensembles in language models. 2022. arXiv preprint arXiv:2207.00747.

35.

Yu S, Krishnapuram B, Rosales R, Rao RB. Bayesian co-training. J Mach Learn Res. 2011;12:2649–80.MathSciNet

36.

Zhou Z, Li M. Semi-supervised regression with co-training. In: Proceedings of IJCAI, 2005; pp. 908–916. http://ijcai.org/Proceedings/05/Papers/0689.pdf.

37.

Sonnenburg S, Rätsch G, Schäfer C. A general and efficient multiple kernel learning algorithm. In: Proceedings of NIPS, 2005; pp. 1273–1280. http://papers.nips.cc/paper/2890-a-general-and-efficient-multiple-kernel-learning-algorithm.

38.

Li J, Gui L, Zhou Y, West D, Aloisi C, He Y. Distilling ChatGPT for explainable automated student answer assessment. 2023. arXiv preprint arXiv:2305.12962.

39.

Lei T, Barzilay R, Jaakkola TS. Rationalizing neural predictions. In: Su, J., Carreras, X., Duh, K. (eds.) Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1-4, 2016, 2016; pp. 107–117. https://doi.org/10.18653/v1/d16-1011.

40.

Du M, Liu N, Yang F, Hu X. Learning credible deep neural networks with rationale regularization. In: Wang, J., Shim, K., Wu, X. (eds.) 2019 IEEE International Conference on Data Mining, ICDM 2019, Beijing, China, November 8-11, 2019, 2019; pp. 150–159. https://doi.org/10.1109/ICDM.2019.00025.

41.

Jiang Z, Zhang Y, Yang Z, Zhao J, Liu K. Alignment rationale for natural language inference. In: Zong, C., Xia, F., Li, W., Navigli, R. (eds.) Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL/IJCNLP 2021, (Volume 1: Long Papers), Virtual Event, August 1-6, 2021, 2021; pp. 5372–5387. https://doi.org/10.18653/v1/2021.acl-long.417.

42.

Vafa K, Deng Y, Blei DM, Rush AM. Rationales for sequential predictions. In: Moens, M., Huang, X., Specia, L., Yih, S.W. (eds.) Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 7-11 November, 2021, 2021; pp. 10314–10332. https://doi.org/10.18653/v1/2021.emnlp-main.807.

43.

Wiegreffe S, Marasović A, Smith NA. Measuring association between labels and free-text rationales. 2020. arXiv preprint arXiv:2010.12762.

Title: Multi-View Cooperative Learning with Invariant Rationale for Document-Level Relation Extraction
Authors: Rui Lin
Jing Fan
Yinglong He
Yehui Yang
Jia Li
Cunhan Guo
Publication date: 27-07-2024
Publisher: Springer US
Published in: Cognitive Computation
Print ISSN: 1866-9956
Electronic ISSN: 1866-9964
DOI: https://doi.org/10.1007/s12559-024-10322-z

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"