nach oben

Erschienen in:

2024 | OriginalPaper | Buchkapitel

Masking Language Model Mechanism with Event-Driven Knowledge Graphs for Temporal Relations Extraction from Clinical Narratives

verfasst von : Kanimozhi Uma, Sumam Francis, Marie-Francine Moens

Erschienen in: Complex Networks & Their Applications XII

Verlag: Springer Nature Switzerland

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

For many natural language processing systems, the extraction of temporal links and associations from clinical narratives has been a critical challenge. To understand such processes, we must be aware of the occurrences of events and their time or temporal aspect by constructing a chronology for the sequence of events. The primary objective of temporal relation extraction is to identify relationships and correlations between entities, events, and expressions. We propose a novel architecture leveraging Transformer based graph neural network by combining textual data with event graph embeddings for predicting temporal links across events, entities, document creation time and expressions. We demonstrate our preliminary findings on i2b2 temporal relations corpus for predicting BEFORE, AFTER and OVERLAP links with event graph for correct set of relations. Comparison with various Biomedical-BERT embedding types were benchmarked yielding best performance on PubMed BERT with language model masking (LMM) mechanism on our methodology. This illustrates the effectiveness of our proposed strategy.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Vorheriges Kapitel A Framework for Empirically Evaluating Pretrained Link Prediction Models

Nächstes Kapitel Efficient Approach for Patient Monitoring: ML-Enabled Framework with Smart Connected Systems

emilyalsentzer/Bio_ClinicalBERT.

microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract-fulltext.

bionlp/bluebert_pubmed_uncased_L-24_H-1024_A-16.

allenai/scibert_scivocab_uncased.

Models - Hugging Face

Alfattni, G., Peek, N., Nenadic, G.: Extraction of temporal relations from clinical free text: a systematic review of current approaches. J. Biomed. Inform. 108(2020), 103488 (2020). https://doi.org/10.1016/j.jbi.2020.103488CrossRef

Alfattni, G., Peek, N., Nenadic, G.: Attention-based bidirectional long short-term memory networks for extracting temporal relationships from clinical discharge summaries. J. Biomed. Inform. 123(2021), 103915 (2021). https://doi.org/10.1016/j.jbi.2021.103915CrossRef

Galvan, D., Okazaki, N., Matsuda, K., Inui, K.: Investigating the challenges of temporal relation extraction from clinical text. In: Lavelli, A., Minard, A.-L., Rinaldi, F. (eds.) Proceedings of the Ninth International Workshop on Health Text Mining and Information Analysis, Louhi@EMNLP 2018, Brussels, Belgium, 31 October 2018, pp. 55–64. Association for Computational Linguistics (2018). https://doi.org/10.18653/v1/w18-5607

Guan, H., Li, J., Xu, H., Devarakonda, M.V.: Robustly pre-trained neural model for direct temporal relation extraction. In: 9th IEEE International Conference on Healthcare Informatics, ICHI 2021, Victoria, BC, Canada, 9–12 August 2021, pp. 501–502. IEEE (2021). https://doi.org/10.1109/ICHI52183.2021.00090

Gumiel, Y.B., et al.: Temporal relation extraction in clinical texts: a systematic review. ACM Comput. Surv. 54(7), 144:1–144:36 (2022). https://doi.org/10.1145/3462475

Han, R., Hsu, I.-H., Yang, M., Galstyan, A., Weischedel, R.M., Peng, N.: Deep structured neural network for event temporal relation extraction. In: Bansal, M., Villavicencio, A. (eds.) Proceedings of the 23rd Conference on Computational Natural Language Learning, CoNLL 2019, Hong Kong, China, 3–4 November 2019, pp. 666–106. Association for Computational Linguistics (2019). https://doi.org/10.18653/v1/K19-1062

Han, R., Ning, Q., Peng, N.: Joint event and temporal relation extraction with shared representations and structured prediction. In: Inui, K., Jiang, J., Ng, V., Wan, X. (eds.) Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, 3–7 November 2019, pp. 434–444. Association for Computational Linguistics (2019). https://doi.org/10.18653/v1/D19-1041

Ul Haq, H., Kocaman, V., Talby, D.: Deeper clinical document understanding using relation extraction. CoRR abs/2112.13259 (2021). arXiv:2112.13259

Lee, H.-J., Zhang, Y., Jiang, M., Xu, J., Tao, C., Xu, H.: Identifying direct temporal relations between time and events from clinical notes. BMC Med. Inform. Decis. Mak. 18(S-2), 23–34 (2018). https://doi.org/10.1186/s12911-018-0627-5

10.

Leeuwenberg, A., Moens, M.-F.: Structured learning for temporal relation extraction from clinical records. In: Lapata, M., Blunsom, P., Koller, A. (eds.) Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2017, Volume 1: Long Papers, Valencia, Spain, 3–7 April 2017, pp. 1150–1158. Association for Computational Linguistics (2017). https://doi.org/10.18653/v1/e17-1108

11.

Leeuwenberg, A., Moens, M.-F.: Temporal information extraction by predicting relative time-lines. In: Riloff, E., Chiang, D., Hockenmaier, J., Tsujii, J. (eds.) Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, 31 October–4 November 2018, pp. 1237–1246. Association for Computational Linguistics (2018). https://doi.org/10.18653/v1/d18-1155

12.

Lin, C., Miller, T., Dligach, D., Bethard, S., Savova, G.: A BERT-based universal model for both within-and cross-sentence clinical temporal relation extraction. In: Proceedings of the 2nd Clinical Natural Language Processing Workshop, pp. 65–71 (2019)

13.

Lin, C., Miller, T., Dligach, D., Sadeque, F., Bethard, S., Savova, G.: A BERT-based one-pass multi-task model for clinical temporal relation extraction (2020)

14.

Lin, C., Miller, T.A., Dligach, D., Amiri, H., Bethard, S., Savova, G.: Self-training improves recurrent neural networks performance for temporal relation extraction. In: Lavelli, A., Minard, A.-L., Rinaldi, F. (eds.) Proceedings of the Ninth International Workshop on Health Text Mining and Information Analysis, Louhi@EMNLP 2018, Brussels, Belgium, 31 October 2018, pp. 165–176. Association for Computational Linguistics (2018). https://doi.org/10.18653/v1/w18-5619

15.

Lin, C., Miller, T.A., Dligach, D., Bethard, S., Savova, G.: Representations of time expressions for temporal relation extraction with convolutional neural networks. In: Cohen, K.B., Demner-Fushman, D., Ananiadou, S., Tsujii, J. (eds.) BioNLP 2017, Vancouver, Canada, 4 August 2017, pp. 322–327. Association for Computational Linguistics (2017). https://doi.org/10.18653/v1/W17-2341

16.

Lin, C., Miller, T.A., Dligach, D., Bethard, S., Savova, G.: EntityBERT: entity-centric masking strategy for model pretraining for the clinical domain. In: Demner-Fushman, D., Cohen, K.B., Ananiadou, S., Tsujii, J. (eds.) Proceedings of the 20th Workshop on Biomedical Language Processing, BioNLP@NAACL-HLT 2021, Online, 11 June 2021, pp. 191–201. Association for Computational Linguistics (2021). https://doi.org/10.18653/v1/2021.bionlp-1.21

17.

Man, H., Ngo, N.T., Van, L.N., Nguyen, T.H.: Selecting optimal context sentences for event-event relation extraction. In: Thirty-Sixth AAAI Conference on Artificial Intelligence, AAAI 2022, Thirty-Fourth Conference on Innovative Applications of Artificial Intelligence, IAAI 2022, The Twelveth Symposium on Educational Advances in Artificial Intelligence, EAAI 2022, Virtual Event, 22 February–1 March 2022, pp. 11058–11066. AAAI Press (2022). https://ojs.aaai.org/index.php/AAAI/article/view/21354

18.

Mathur, P., Jain, R., Dernoncourt, F., Morariu, V.I., Tran, Q.H., Manocha, D.: TIMERS: document-level temporal relation extraction. In: Zong, C., Xia, F., Li, W., Navigli, R. (eds.) Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL/IJCNLP 2021, Volume 2: Short Papers, Virtual Event, 1–6 August 2021, pp. 524–533. Association for Computational Linguistics (2021). https://doi.org/10.18653/v1/2021.acl-short.67

19.

Ning, Q., Feng, Z., Roth, D.: A structured learning approach to temporal relation extraction. In: Palmer, M., Hwa, R., Riedel, S. (eds.) Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP 2017, Copenhagen, Denmark, 9–11 September 2017, pp. 1027–1037. Association for Computational Linguistics (2017). https://doi.org/10.18653/v1/d17-1108

20.

Peng, Y., Yan, S., Lu, Z.: Transfer learning in biomedical natural language processing: an evaluation of BERT and ELMo on ten benchmarking datasets. In: Demner-Fushman, D., Cohen, K.B., Ananiadou, S., Tsujii, J. (eds.) Proceedings of the 18th BioNLP Workshop and Shared Task, BioNLP@ACL 2019, Florence, Italy, 1 August 2019, pp. 58–65. Association for Computational Linguistics (2019). https://doi.org/10.18653/v1/w19-5006

21.

Pennington, J., Socher, R., Manning, C.D.: GloVe: global vectors for word representation. In: Moschitti, A., Pang, B., Daelemans, W., (eds.) Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, A meeting of SIGDAT, a Special Interest Group of the ACL, 25–29 October 2014, Doha, Qatar, pp. 1532–1543. ACL (2014). https://doi.org/10.3115/v1/d14-1162

22.

Schlichtkrull, M., Kipf, T.N., Bloem, P., van den Berg, R., Titov, I., Welling, M.: Modeling relational data with graph convolutional networks. In: Gangemi, A., et al. (eds.) ESWC 2018. LNCS, vol. 10843, pp. 593–607. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-93417-4_38CrossRef

23.

Sun, W., Rumshisky, A., Uzuner, O.: Evaluating temporal relations in clinical text: 2012 i2b2 challenge. J. Am. Med. Inf. Assoc. 20(5), 806–813 (2013). https://doi.org/10.1136/amiajnl-2013-001628CrossRef

24.

Tourille, J., Ferret, O., Névéol, A., Tannier, X.: Neural architecture for temporal relation extraction: a Bi-LSTM approach for detecting narrative containers. In: Barzilay, R., Kan, M.-Y. (eds.) Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Volume 2: Short Papers, Vancouver, Canada, 30 July–4 August, pp. 224–230. Association for Computational Linguistics (2017). https://doi.org/10.18653/v1/P17-2035

25.

Wang, L., Li, P., Xu, S.: DCT-centered temporal relation extraction. In: Calzolari, N., et al. (eds.) Proceedings of the 29th International Conference on Computational Linguistics, COLING 2022, Gyeongju, Republic of Korea, 12–17 October 2022, pp. 2087–2097. International Committee on Computational Linguistics (2022). https://aclanthology.org/2022.coling-1.182

26.

Wang, M., et al.: Deep graph library: a graph-centric, highly-performant package for graph neural networks. arXiv preprint arXiv:1909.01315 (2019)

27.

Zhang, S., Ning, Q., Huang, L.: Extracting temporal event relation with syntax-guided graph transformer. In: Carpuat, M., de Marneffe, M.-C., Ruíz, I.V.M. (eds.) Findings of the Association for Computational Linguistics, NAACL 2022, Seattle, WA, United States, 10–15 July 2022, pp. 379–390. Association for Computational Linguistics (2022). https://doi.org/10.18653/v1/2022.findings-naacl.29

28.

Zhao, X., Lin, S.-T., Durrett, G.: Effective distant supervision for temporal relation extraction. CoRR abs/2010.12755. arXiv arXiv:2010.12755 (2020)

29.

Zhou, Y., et al.: Clinical temporal relation extraction with probabilistic soft logic regularization and global inference. In: Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, 2–9 February 2021, pp. 14647–14655. AAAI Press (2021). https://ojs.aaai.org/index.php/AAAI/article/view/177212023-09-1109:46

Titel: Masking Language Model Mechanism with Event-Driven Knowledge Graphs for Temporal Relations Extraction from Clinical Narratives
verfasst von: Kanimozhi Uma
Sumam Francis
Marie-Francine Moens
Verlag: Springer Nature Switzerland
Buch: Complex Networks & Their Applications XII
Print ISBN: 978-3-031-53467-6

Electronic ISBN: 978-3-031-53468-3

Copyright-Jahr: 2024
DOI: https://doi.org/10.1007/978-3-031-53468-3_14

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partner