Top

Hint

Swipe to navigate through the chapters of this book

Published in:

Read chapter Read first chapter

2021 | OriginalPaper | Chapter

Multi-view Analysis of Unregistered Medical Images Using Cross-View Transformers

Authors : Gijs van Tulder, Yao Tong, Elena Marchiori

Published in: Medical Image Computing and Computer Assisted Intervention – MICCAI 2021

Publisher: Springer International Publishing

Abstract

Multi-view medical image analysis often depends on the combination of information from multiple views. However, differences in perspective or other forms of misalignment can make it difficult to combine views effectively, as registration is not always possible. Without registration, views can only be combined at a global feature level, by joining feature vectors after global pooling. We present a novel cross-view transformer method to transfer information between unregistered views at the level of spatial feature maps. We demonstrate this method on multi-view mammography and chest X-ray datasets. On both datasets, we find that a cross-view transformer that links spatial feature maps can outperform a baseline model that joins feature vectors after global pooling.

To get access to this content you need the following product:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 69.000 Bücher
über 500 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt 90 Tage mit der neuen Mini-Lizenz testen!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 50.000 Bücher
über 380 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt 90 Tage mit der neuen Mini-Lizenz testen!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 58.000 Bücher
über 300 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt 90 Tage mit der neuen Mini-Lizenz testen!

inform now

previous chapter Spine-Transformers: Vertebra Detection and Localization in Arbitrary Field-of-View Spine CT with Transformers

next chapter Stain Mix-Up: Unsupervised Domain Generalization for Histopathology Images

Available only for authorised users

The code for the experiments is available at https://vantulder.net/code/2021/miccai-transformers/.

Bekker, A.J., Shalhon, M., Greenspan, H., Goldberger, J.: Multi-view probabilistic classification of breast microcalcifications. IEEE Trans. Med. Imaging 35(2), 645–653 (2015). https://doi.org/10.1109/TMI.2015.2488019 CrossRef

Carneiro, G., Nascimento, J., Bradley, A.P.: Deep learning models for classifying mammogram exams containing unregistered multi-view images and segmentation maps of lesions. In: Zhou, S.K., Greenspan, H., Shen, D. (eds.) Deep Learning for Medical Image Analysis, pp. 321–339. Academic Press (January 2017). https://doi.org/10.1016/B978-0-12-810408-8.00019-5

Hashir, M., Bertrand, H., Cohen, J.P.: Quantifying the value of lateral views in deep learning for chest X-rays. In: Medical Imaging with Deep Learning, pp. 288–303. PMLR (September 2020)

He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR, pp. 770–780 (2016). https://doi.org/10.3389/fpsyg.2013.00124

Heath, M., Bowyer, K., Kopans, D., Moore, R., Kegelmeyer Jr, P.: The Digital Database for Screening Mammography (2001)

Irvin, J., et al.: CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison. arXiv:1901.07031 [cs, eess] (January 2019)

Lee, R.S., Gimenez, F., Hoogi, A., Miyake, K.K., Gorovoy, M., Rubin, D.L.: A curated mammography data set for use in computer-aided detection and diagnosis research. Sci. Data 4, 1–9 (2017). https://doi.org/10.1038/sdata.2017.177

Liu, Y., Zhang, F., Zhang, Q., Wang, S., Wang, Y., Yu, Y.: Cross-view correspondence reasoning based on bipartite graph convolutional network for mammogram mass detection. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3811–3821. IEEE, Seattle, WA, USA (June 2020). https://doi.org/10.1109/CVPR42600.2020.00387

Ma, J., et al.: Cross-view Relation Networks for Mammogram Mass Detection. arXiv:1907.00528 [cs] (June 2019)

10.

Nasir Khan, H., Shahid, A.R., Raza, B., Dar, A.H., Alquhayz, H.: Multi-view feature fusion based four views model for mammogram classification using convolutional neural network. IEEE Access 7, 165724–165733 (2019). https://doi.org/10.1109/ACCESS.2019.2953318 CrossRef

11.

Rubin, J., Sanghavi, D., Zhao, C., Lee, K., Qadir, A., Xu-Wilson, M.: Large Scale Automated Reading of Frontal and Lateral Chest X-Rays using Dual Convolutional Neural Networks. arXiv:1804.07839 [cs, stat] (April 2018)

12.

Sun, L., Wang, J., Hu, Z., Xu, Y., Cui, Z.: Multi-view convolutional neural networks for mammographic image classification. IEEE Access 7, 126273–126282 (2019). https://doi.org/10.1109/ACCESS.2019.2939167 CrossRef

13.

Vaswani, A., et al.: Attention is all you need. Adv. Neural. Inf. Process. Syst. 30, 5998–6008 (2017)

14.

Wang, H., et al.: Breast mass classification via deeply integrating the contextual information from multi-view data. Pattern Recogn. 80, 42–52 (2018). https://doi.org/10.1016/j.patcog.2018.02.026 CrossRef

15.

Wu, B., et al.: Visual Transformers: Token-based Image Representation and Processing for Computer Vision. arXiv:2006.03677 [cs, eess] (November 2020)

16.

Wu, N., et al.: Deep neural networks improve radiologists’ performance in breast cancer screening. IEEE Trans. Med. Imaging 39(4), 1184–1194 (2019). https://doi.org/10.1109/tmi.2019.2945514 CrossRef

17.

Zhao, X., Yu, L., Wang, X.: Cross-view attention network for breast cancer screening from multi-view mammograms. In: ICASSP 2020–2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1050–1054 (May 2020). https://doi.org/10.1109/ICASSP40776.2020.9054612

Title: Multi-view Analysis of Unregistered Medical Images Using Cross-View Transformers
Authors: Gijs van Tulder
Yao Tong
Elena Marchiori
Publisher: Springer International Publishing
Book: Medical Image Computing and Computer Assisted Intervention – MICCAI 2021
Print ISBN: 978-3-030-87198-7

Electronic ISBN: 978-3-030-87199-4

Copyright Year: 2021
DOI: https://doi.org/10.1007/978-3-030-87199-4_10

Springer Professional

Hint

Swipe to navigate through the chapters of this book

Abstract

Please log in to get access to this content

To get access to this content you need the following product:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partner