Skip to main content
Top

2023 | OriginalPaper | Chapter

BERT for Complex Systematic Review Screening to Support the Future of Medical Research

Authors : Marta Hasny, Alexandru-Petru Vasile, Mario Gianni, Alexandra Bannach-Brown, Mona Nasser, Murray Mackay, Diana Donovan, Jernej Šorli, Ioana Domocos, Milad Dulloo, Nimita Patel, Olivia Drayson, Nicole Meerah Elango, Jéromine Vacquie, Ana Patricia Ayala, Anna Fogtman

Published in: Artificial Intelligence in Medicine

Publisher: Springer Nature Switzerland

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

This work presents a Natural Language Processing approach to screen complex datasets of medical articles to provide timely and efficient response to pressing issues in medicine. The approach is based on the Bidirectional Encoder Representation from Transformers (BERT) to screen the articles using their titles and abstracts. Systematic review screening is a classification task aiming at selecting articles fulfilling the criteria for the next step of the review. A number of BERT models are fine-tuned for this classification task. Two challenging space medicine systematic review datasets that include human, animal, and in-vitro studies are used for the evaluation of the models. Backtranslation is used as a data augmentation technique to handle the class imbalance and a performance comparison of the models on the original and augmented data is presented. The BERT models provide an accessible solution for screening systematic reviews, which are considered complex and time-consuming. The proposed approach can change the workflow of conducting these types of reviews, especially in response to urgent policy and practice questions in medicine. The source code and datasets are available on GitHub: https://​github.​com/​ESA-RadLab/​BERTCSRS.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
2.
go back to reference Aum, S., Choe, S.: srBERT: automatic article classification model for systematic review using BERT. Syst. Rev. 10(1), 1–8 (2021)CrossRef Aum, S., Choe, S.: srBERT: automatic article classification model for systematic review using BERT. Syst. Rev. 10(1), 1–8 (2021)CrossRef
5.
go back to reference Beltagy, I., Lo, K., Cohan, A.: SciBERT: a pretrained language model for scientific text. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Association for Computational Linguistics, Stroudsburg, PA, USA (2019) Beltagy, I., Lo, K., Cohan, A.: SciBERT: a pretrained language model for scientific text. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Association for Computational Linguistics, Stroudsburg, PA, USA (2019)
6.
go back to reference Borah, R., Brown, A.W., Capers, P.L., Kaiser, K.A.: Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the prospero registry. BMJ Open 7(2), e012545 (2017)CrossRef Borah, R., Brown, A.W., Capers, P.L., Kaiser, K.A.: Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the prospero registry. BMJ Open 7(2), e012545 (2017)CrossRef
7.
go back to reference Chawla, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P.: SMOTE: synthetic minority over-sampling technique. J. Artif. Intell. Res. 16, 321–357 (2002)CrossRefMATH Chawla, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P.: SMOTE: synthetic minority over-sampling technique. J. Artif. Intell. Res. 16, 321–357 (2002)CrossRefMATH
11.
go back to reference Gu, Y., et al.: Domain-specific language model pretraining for biomedical natural language processing (2020) Gu, Y., et al.: Domain-specific language model pretraining for biomedical natural language processing (2020)
12.
go back to reference Higgins, J., et al.: Cochrane handbook for systematic reviews of interventions version 6.3 (updated February 2022). Cochrane, www.training.cochrane.org/handbook (2022) Higgins, J., et al.: Cochrane handbook for systematic reviews of interventions version 6.3 (updated February 2022). Cochrane, www.training.cochrane.org/handbook (2022)
17.
go back to reference Lee, J., et al.: BioBERT: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics 36(4), 1234–1240 (2020)CrossRef Lee, J., et al.: BioBERT: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics 36(4), 1234–1240 (2020)CrossRef
21.
go back to reference Paszke, A., et al.: Automatic differentiation in PyTorch. In: NIPS-W (2017) Paszke, A., et al.: Automatic differentiation in PyTorch. In: NIPS-W (2017)
22.
go back to reference Sennrich, R., Haddow, B., Birch, A.: Improving neural machine translation models with monolingual data. arXiv preprint arXiv:1511.06709 (2015) Sennrich, R., Haddow, B., Birch, A.: Improving neural machine translation models with monolingual data. arXiv preprint arXiv:​1511.​06709 (2015)
23.
24.
26.
go back to reference Wallace, B.C., Trikalinos, T.A., Lau, J., Brodley, C., Schmid, C.H.: Semi-automated screening of biomedical citations for systematic reviews. BMC Bioinform. 11(1), 1–11 (2010)CrossRef Wallace, B.C., Trikalinos, T.A., Lau, J., Brodley, C., Schmid, C.H.: Semi-automated screening of biomedical citations for systematic reviews. BMC Bioinform. 11(1), 1–11 (2010)CrossRef
27.
go back to reference Wieting, J., Mallinson, J., Gimpel, K.: Learning paraphrastic sentence embeddings from back-translated bitext. arXiv preprint arXiv:1706.01847 (2017) Wieting, J., Mallinson, J., Gimpel, K.: Learning paraphrastic sentence embeddings from back-translated bitext. arXiv preprint arXiv:​1706.​01847 (2017)
29.
go back to reference Yu, A.W., Dohan, D., Luong, M.T., Zhao, R., Chen, K., Norouzi, M., Le, Q.V.: QANet: combining local convolution with global self-attention for reading comprehension (2018) Yu, A.W., Dohan, D., Luong, M.T., Zhao, R., Chen, K., Norouzi, M., Le, Q.V.: QANet: combining local convolution with global self-attention for reading comprehension (2018)
Metadata
Title
BERT for Complex Systematic Review Screening to Support the Future of Medical Research
Authors
Marta Hasny
Alexandru-Petru Vasile
Mario Gianni
Alexandra Bannach-Brown
Mona Nasser
Murray Mackay
Diana Donovan
Jernej Šorli
Ioana Domocos
Milad Dulloo
Nimita Patel
Olivia Drayson
Nicole Meerah Elango
Jéromine Vacquie
Ana Patricia Ayala
Anna Fogtman
Copyright Year
2023
DOI
https://doi.org/10.1007/978-3-031-34344-5_21

Premium Partner