Skip to main content

2020 | OriginalPaper | Buchkapitel

Using Augmented Reality and Machine Learning in Radiology

verfasst von : Lucian Trestioreanu, Patrick Glauner, Jorge Augusto Meira, Max Gindt, Radu State

Erschienen in: Innovative Technologies for Market Leadership

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Surgeries are one of the main cost factors of health care systems. To reduce the costs related to diagnoses and surgeries, we propose a system for automated segmentation of medical images in order to segment body parts like liver or lesions. The model is based on convolutional neural networks, for which we show promising results on real computed tomography scans. The deep learning algorithm is part of a larger system that aims to support doctors by visualizing the segments in a Microsoft HoloLens, an augmented reality device. Our approach allows doctors to intuitively look at and interact with the holographic data rather than using 2D screens, enabling them to provide better health care. Both the machine learning algorithm and the visualization utilize high-performance GPUs in order to enable doctors to interact efficiently with our system.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
Zurück zum Zitat Aljabar, P., Heckemann, R. A., Hammers, A., Hajnal, J.V., & Rueckert, D. (2009). Multi-atlas based segmentation of brain images: Atlas selection and its effect on accuracy. NeuroImage, 46(3), 726–738.CrossRef Aljabar, P., Heckemann, R. A., Hammers, A., Hajnal, J.V., & Rueckert, D. (2009). Multi-atlas based segmentation of brain images: Atlas selection and its effect on accuracy. NeuroImage, 46(3), 726–738.CrossRef
Zurück zum Zitat Bellver, M., Maninis, K.-K., Pont-Tuset, J., Giró-i-Nieto, X., Torres, J., & Van Gool, L. (2017). Detection-aided liver lesion segmentation using deep learning. arXiv:1711.11069v1. Bellver, M., Maninis, K.-K., Pont-Tuset, J., Giró-i-Nieto, X., Torres, J., & Van Gool, L. (2017). Detection-aided liver lesion segmentation using deep learning. arXiv:1711.11069v1.
Zurück zum Zitat Beydoun, A., Gupta, V., & Siegel, E. (2017). DICOM to 3D Holograms: Use Case for Augmented Reality in Diagnostic and Interventional Radiology. In SIIM scientific session posters and demonstrations. Beydoun, A., Gupta, V., & Siegel, E. (2017). DICOM to 3D Holograms: Use Case for Augmented Reality in Diagnostic and Interventional Radiology. In SIIM scientific session posters and demonstrations.
Zurück zum Zitat Calhoun, P., Kuszyk, B., Heath, D., Carley, J., & Fishman, E. (1999). Three-dimensional volume rendering of spiral CT data: Theory and method. Radiographics: A Review Publication of the Radiological Society of North America, Inc., 19(3), 745–764.CrossRef Calhoun, P., Kuszyk, B., Heath, D., Carley, J., & Fishman, E. (1999). Three-dimensional volume rendering of spiral CT data: Theory and method. Radiographics: A Review Publication of the Radiological Society of North America, Inc., 19(3), 745–764.CrossRef
Zurück zum Zitat Carmigniani, J., & Furht, B. (2011). Augmented reality: An overview. In B. Furht (Ed.) Handbook of Augmented Reality. New York: Springer. Carmigniani, J., & Furht, B. (2011). Augmented reality: An overview. In B. Furht (Ed.) Handbook of Augmented Reality. New York: Springer.
Zurück zum Zitat Christ, P., Ettlinger, F., Grun, F., Elshaera, M. E. A., Lipkova, J., Schlecht, S., et al. (2017). Automatic liver and tumor segmentation of CT and MRI volumes using cascaded fully convolutional neural networks. arXiv 1702.05970. Christ, P., Ettlinger, F., Grun, F., Elshaera, M. E. A., Lipkova, J., Schlecht, S., et al. (2017). Automatic liver and tumor segmentation of CT and MRI volumes using cascaded fully convolutional neural networks. arXiv 1702.05970.
Zurück zum Zitat Douglas, D., Venets, D., Wilke, C., Gibson, D., Liotta, L., Petricoin, E., et al. (2018). Augmented reality and virtual reality: initial successes in diagnostic radiology. In State of the art virtual reality and augmented reality knowhow. London: IntechOpen Limited. Douglas, D., Venets, D., Wilke, C., Gibson, D., Liotta, L., Petricoin, E., et al. (2018). Augmented reality and virtual reality: initial successes in diagnostic radiology. In State of the art virtual reality and augmented reality knowhow. London: IntechOpen Limited.
Zurück zum Zitat Douglas, D., Wilke, C., Gibson, D., Boone, J., & Wintermark, M. (2017). Augmented reality: advances in diagnostic imaging. Multimodal Technologies and Interact, 1, 29.CrossRef Douglas, D., Wilke, C., Gibson, D., Boone, J., & Wintermark, M. (2017). Augmented reality: advances in diagnostic imaging. Multimodal Technologies and Interact, 1, 29.CrossRef
Zurück zum Zitat Fishman, E., Ney, D., Heath, D., Corl, F., Horton, K., & Johnson, P. (2006). Volume rendering versus maximum intensity projection in CT angiography: What works best, when, and why. Radiographics: A Review Publication of the Radiological Society of North America, Inc., 26(3), 905–922.CrossRef Fishman, E., Ney, D., Heath, D., Corl, F., Horton, K., & Johnson, P. (2006). Volume rendering versus maximum intensity projection in CT angiography: What works best, when, and why. Radiographics: A Review Publication of the Radiological Society of North America, Inc., 26(3), 905–922.CrossRef
Zurück zum Zitat Glauner, P., Migliosi, A., Meira, J. A., Valtchev, P., State, R., & Bettinger, B. (2017). Is big data sufficient for a reliable detection of non-technical losses? In 2017 19th International Conference on Intelligent System Application to Power Systems (ISAP). Glauner, P., Migliosi, A., Meira, J. A., Valtchev, P., State, R., & Bettinger, B. (2017). Is big data sufficient for a reliable detection of non-technical losses? In 2017 19th International Conference on Intelligent System Application to Power Systems (ISAP).
Zurück zum Zitat Glauner, P., State, R., Valtchev, P., & Duarte, D. (2018a). On the reduction of biases in big data sets for the detection of irregular power usage. In Proceedings of the 13th International FLINS Conference on Data Science and Knowledge Engineering for Sensing Decision Support (FLINS 2018). Glauner, P., State, R., Valtchev, P., & Duarte, D. (2018a). On the reduction of biases in big data sets for the detection of irregular power usage. In Proceedings of the 13th International FLINS Conference on Data Science and Knowledge Engineering for Sensing Decision Support (FLINS 2018).
Zurück zum Zitat Glauner, P., Valtchev, P., & State, R. (2018b). Impact of biases in big data. In Proceedings of the 26th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN 2018). Glauner, P., Valtchev, P., & State, R. (2018b). Impact of biases in big data. In Proceedings of the 26th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN 2018).
Zurück zum Zitat Gotra, A., Sivakumaran, L., Chartrand, G., Vu, K. N., Vandenbroucke-Menu, F., Kauffmann, C., et al. (2017). Liver segmentation: indications, techniques and future directions. Insights Imaging, 8(4), 377–392.CrossRef Gotra, A., Sivakumaran, L., Chartrand, G., Vu, K. N., Vandenbroucke-Menu, F., Kauffmann, C., et al. (2017). Liver segmentation: indications, techniques and future directions. Insights Imaging, 8(4), 377–392.CrossRef
Zurück zum Zitat Heath, D., Corl, F., Horton, K., Fishman, E., & Johnson, P. (2006). Volume rendering versus maximum intensity projection in CT angiography: What works best, when, and why. Radiographics: A Review Publication of the Radiological Society of North America, Inc., 26(3), 905–922.CrossRef Heath, D., Corl, F., Horton, K., Fishman, E., & Johnson, P. (2006). Volume rendering versus maximum intensity projection in CT angiography: What works best, when, and why. Radiographics: A Review Publication of the Radiological Society of North America, Inc., 26(3), 905–922.CrossRef
Zurück zum Zitat Hohne, K., Bomans, M., Tiede, U., & Riemer, M. (1988). Display of multiple 3D-objects using the generalized voxel-model. In Medical Imaging II. Newport Beach: SPIE. Hohne, K., Bomans, M., Tiede, U., & Riemer, M. (1988). Display of multiple 3D-objects using the generalized voxel-model. In Medical Imaging II. Newport Beach: SPIE.
Zurück zum Zitat Hoogi, A., Beaulieu, C. F., Cunha, G. M., Heba, E., Sirlin, C. B., Napel, S., et al. (2017a). Adaptive local window for level set segmentation of CT and MRI liver lesions. Medical Image Analysis, 37, 46–55.CrossRef Hoogi, A., Beaulieu, C. F., Cunha, G. M., Heba, E., Sirlin, C. B., Napel, S., et al. (2017a). Adaptive local window for level set segmentation of CT and MRI liver lesions. Medical Image Analysis, 37, 46–55.CrossRef
Zurück zum Zitat Hoogi, A., Lambert, J. W., Zheng, Y., Comaniciu, D., & Rubin, D. L. (2017b). A fully automated pipeline for detection and segmentation of liver lesions and pathological lymph nodes. arXiv 1703.06418. Hoogi, A., Lambert, J. W., Zheng, Y., Comaniciu, D., & Rubin, D. L. (2017b). A fully automated pipeline for detection and segmentation of liver lesions and pathological lymph nodes. arXiv 1703.06418.
Zurück zum Zitat Johnson, P., Heath, D., Kuszyk, B., & Fishman, E. (1996). CT angiography with volume rendering: Advantages and applications in splanchnic vascular imaging. Radiology, 200(2), 564–568.CrossRef Johnson, P., Heath, D., Kuszyk, B., & Fishman, E. (1996). CT angiography with volume rendering: Advantages and applications in splanchnic vascular imaging. Radiology, 200(2), 564–568.CrossRef
Zurück zum Zitat Jung, T. (2019). The power of AR and VR for business. In M. Claudia, T. Dieck, & T. Jung (Eds.) Augmented reality and virtual reality. Berlin: Springer. Jung, T. (2019). The power of AR and VR for business. In M. Claudia, T. Dieck, & T. Jung (Eds.) Augmented reality and virtual reality. Berlin: Springer.
Zurück zum Zitat Maninis, K.-K., Pont-Tuset, J., Arbelaez, P., Van Gool, L. (2016). Deep retinal image understanding. In International Conference on Medical Image Computing and Computer-Assisted Intervention (pp. 140–148). Berlin: Springer. Maninis, K.-K., Pont-Tuset, J., Arbelaez, P., Van Gool, L. (2016). Deep retinal image understanding. In International Conference on Medical Image Computing and Computer-Assisted Intervention (pp. 140–148). Berlin: Springer.
Zurück zum Zitat Ronneberger, O., Fischer, P., & Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention (pp. 234–241). Berlin: Springer. Ronneberger, O., Fischer, P., & Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention (pp. 234–241). Berlin: Springer.
Zurück zum Zitat Rothman, D. (2018). Artificial intelligence by example: Develop machine intelligence from scratch using real artificial intelligence use cases. Birmingham: Packt Publishing. Rothman, D. (2018). Artificial intelligence by example: Develop machine intelligence from scratch using real artificial intelligence use cases. Birmingham: Packt Publishing.
Metadaten
Titel
Using Augmented Reality and Machine Learning in Radiology
verfasst von
Lucian Trestioreanu
Patrick Glauner
Jorge Augusto Meira
Max Gindt
Radu State
Copyright-Jahr
2020
DOI
https://doi.org/10.1007/978-3-030-41309-5_8