Skip to main content
Erschienen in:
Buchtitelbild

2020 | OriginalPaper | Buchkapitel

Reconstructing Sinus Anatomy from Endoscopic Video – Towards a Radiation-Free Approach for Quantitative Longitudinal Assessment

verfasst von : Xingtong Liu, Maia Stiber, Jindan Huang, Masaru Ishii, Gregory D. Hager, Russell H. Taylor, Mathias Unberath

Erschienen in: Medical Image Computing and Computer Assisted Intervention – MICCAI 2020

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Reconstructing accurate 3D surface models of sinus anatomy directly from an endoscopic video is a promising avenue for cross-sectional and longitudinal analysis to better understand the relationship between sinus anatomy and surgical outcomes. We present a patient-specific, learning-based method for 3D reconstruction of sinus surface anatomy directly and only from endoscopic videos. We demonstrate the effectiveness and accuracy of our method on in and ex vivo data where we compare to sparse reconstructions from Structure from Motion, dense reconstruction from COLMAP, and ground truth anatomy from CT. Our textured reconstructions are watertight and enable measurement of clinically relevant parameters in good agreement with CT. The source code is available at https://​github.​com/​lppllppl920/​DenseReconstruct​ion-Pytorch.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Anhänge
Nur mit Berechtigung zugänglich
Literatur
1.
Zurück zum Zitat Bezerra, T.F.P., et al.: Quality of life assessment septoplasty in patients with nasal obstruction. Braz. J. Otorhinolaryngol. 78(3), 57–62 (2012)CrossRef Bezerra, T.F.P., et al.: Quality of life assessment septoplasty in patients with nasal obstruction. Braz. J. Otorhinolaryngol. 78(3), 57–62 (2012)CrossRef
2.
Zurück zum Zitat Hytönen, M., Blomgren, K., Lilja, M., Mäkitie, A.: How we do it: septoplasties under local anaesthetic are suitable for short stay surgery; the clinical outcomes. Clin. Otolaryngol.: Official J. ENT-UK; Official J. Netherlands Soc. Oto-Rhino-Laryngol. Cervico-Facial Surg. 31(1), 64–68 (2006) Hytönen, M., Blomgren, K., Lilja, M., Mäkitie, A.: How we do it: septoplasties under local anaesthetic are suitable for short stay surgery; the clinical outcomes. Clin. Otolaryngol.: Official J. ENT-UK; Official J. Netherlands Soc. Oto-Rhino-Laryngol. Cervico-Facial Surg. 31(1), 64–68 (2006)
3.
Zurück zum Zitat Hytönen, M.L., Lilja, M., Mäkitie, A.A., Sintonen, H., Roine, R.P.: Does septoplasty enhance the quality of life in patients? Eur. Arch. Otorhinolaryngol. 269(12), 2497–2503 (2012)CrossRef Hytönen, M.L., Lilja, M., Mäkitie, A.A., Sintonen, H., Roine, R.P.: Does septoplasty enhance the quality of life in patients? Eur. Arch. Otorhinolaryngol. 269(12), 2497–2503 (2012)CrossRef
4.
Zurück zum Zitat Schönberger, J.L., Frahm, J.: Structure-from-motion revisited. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4104–4113, June 2016 Schönberger, J.L., Frahm, J.: Structure-from-motion revisited. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4104–4113, June 2016
5.
Zurück zum Zitat Phan, T., Trinh, D., Lamarque, D., Wolf, D., Daul, C.: Dense optical flow for the reconstruction of weakly textured and structured surfaces: application to endoscopy. In: 2019 IEEE International Conference on Image Processing (ICIP), pp. 310–314, September 2019 Phan, T., Trinh, D., Lamarque, D., Wolf, D., Daul, C.: Dense optical flow for the reconstruction of weakly textured and structured surfaces: application to endoscopy. In: 2019 IEEE International Conference on Image Processing (ICIP), pp. 310–314, September 2019
6.
Zurück zum Zitat Widya, A.R., Monno, Y., Okutomi, M., Suzuki, S., Gotoda, T., Miki, K.: Whole stomach 3D reconstruction and frame localization from monocular endoscope video. IEEE J. Transl. Eng. Health Med. 7, 1–10 (2019)CrossRef Widya, A.R., Monno, Y., Okutomi, M., Suzuki, S., Gotoda, T., Miki, K.: Whole stomach 3D reconstruction and frame localization from monocular endoscope video. IEEE J. Transl. Eng. Health Med. 7, 1–10 (2019)CrossRef
7.
Zurück zum Zitat Qiu, L., Ren, H.: Endoscope navigation and 3D reconstruction of oral cavity by visual slam with mitigated data scarcity. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 2278–22787, June 2018 Qiu, L., Ren, H.: Endoscope navigation and 3D reconstruction of oral cavity by visual slam with mitigated data scarcity. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 2278–22787, June 2018
8.
Zurück zum Zitat Kazhdan, M., Bolitho, M., Hoppe, H.: Poisson surface reconstruction. In: Proceedings of the Fourth Eurographics Symposium on Geometry Processing, vol. 7 (2006) Kazhdan, M., Bolitho, M., Hoppe, H.: Poisson surface reconstruction. In: Proceedings of the Fourth Eurographics Symposium on Geometry Processing, vol. 7 (2006)
9.
Zurück zum Zitat Turan, M., Pilavci, Y.Y., Ganiyusufoglu, I., Araujo, H., Konukoglu, E., Sitti, M.: Sparse-then-dense alignment-based 3D map reconstruction method for endoscopic capsule robots. Mach. Vis. Appl. 29(2), 345–359 (2018)CrossRef Turan, M., Pilavci, Y.Y., Ganiyusufoglu, I., Araujo, H., Konukoglu, E., Sitti, M.: Sparse-then-dense alignment-based 3D map reconstruction method for endoscopic capsule robots. Mach. Vis. Appl. 29(2), 345–359 (2018)CrossRef
10.
Zurück zum Zitat Tokgozoglu, H.N., Meisner, E.M., Kazhdan, M., Hager, G.D.: Color-based hybrid reconstruction for endoscopy. In: 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pp. 8–15, June 2012 Tokgozoglu, H.N., Meisner, E.M., Kazhdan, M., Hager, G.D.: Color-based hybrid reconstruction for endoscopy. In: 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pp. 8–15, June 2012
11.
Zurück zum Zitat Karargyris, A., Bourbakis, N.: Three-dimensional reconstruction of the digestive wall in capsule endoscopy videos using elastic video interpolation. IEEE Trans. Med. Imaging 30(4), 957–971 (2011)CrossRef Karargyris, A., Bourbakis, N.: Three-dimensional reconstruction of the digestive wall in capsule endoscopy videos using elastic video interpolation. IEEE Trans. Med. Imaging 30(4), 957–971 (2011)CrossRef
12.
Zurück zum Zitat Zhao, Q., Price, T., Pizer, S., Niethammer, M., Alterovitz, R., Rosenman, J.: The endoscopogram: a 3D model reconstructed from endoscopic video frames. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9900, pp. 439–447. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46720-7_51CrossRef Zhao, Q., Price, T., Pizer, S., Niethammer, M., Alterovitz, R., Rosenman, J.: The endoscopogram: a 3D model reconstructed from endoscopic video frames. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9900, pp. 439–447. Springer, Cham (2016). https://​doi.​org/​10.​1007/​978-3-319-46720-7_​51CrossRef
13.
Zurück zum Zitat Lamarca, J., Parashar, S., Bartoli, A., Montiel, J.: DefSLAM: tracking and mapping of deforming scenes from monocular sequences. arXiv preprint arXiv:1908.08918 (2019) Lamarca, J., Parashar, S., Bartoli, A., Montiel, J.: DefSLAM: tracking and mapping of deforming scenes from monocular sequences. arXiv preprint arXiv:​1908.​08918 (2019)
14.
Zurück zum Zitat Song, J., Wang, J., Zhao, L., Huang, S., Dissanayake, G.: MIS-SLAM: real-time large-scale dense deformable slam system in minimal invasive surgery based on heterogeneous computing. IEEE Robot. Autom. Lett. 3(4), 4068–4075 (2018)CrossRef Song, J., Wang, J., Zhao, L., Huang, S., Dissanayake, G.: MIS-SLAM: real-time large-scale dense deformable slam system in minimal invasive surgery based on heterogeneous computing. IEEE Robot. Autom. Lett. 3(4), 4068–4075 (2018)CrossRef
16.
Zurück zum Zitat Chen, R.J., Bobrow, T.L., Athey, T., Mahmood, F., Durr, N.J.: Slam endoscopy enhanced by adversarial depth prediction. arXiv preprint arXiv:1907.00283 (2019) Chen, R.J., Bobrow, T.L., Athey, T., Mahmood, F., Durr, N.J.: Slam endoscopy enhanced by adversarial depth prediction. arXiv preprint arXiv:​1907.​00283 (2019)
17.
Zurück zum Zitat Liu, X., et al.: Dense depth estimation in monocular endoscopy with self-supervised learning methods. IEEE Trans. Med. Imaging 39, 1438–1447 (2019)CrossRef Liu, X., et al.: Dense depth estimation in monocular endoscopy with self-supervised learning methods. IEEE Trans. Med. Imaging 39, 1438–1447 (2019)CrossRef
18.
Zurück zum Zitat Liu, X., et al.: Extremely dense point correspondences using a learned feature descriptor. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4847–4856 (2020) Liu, X., et al.: Extremely dense point correspondences using a learned feature descriptor. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4847–4856 (2020)
19.
Zurück zum Zitat Zhou, T., Brown, M., Snavely, N., Lowe, D.G.: Unsupervised learning of depth and ego-motion from video. In: CVPR (2017) Zhou, T., Brown, M., Snavely, N., Lowe, D.G.: Unsupervised learning of depth and ego-motion from video. In: CVPR (2017)
20.
Zurück zum Zitat Curless, B., Levoy, M.: A volumetric method for building complex models from range images. In: Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, pp. 303–312 (1996) Curless, B., Levoy, M.: A volumetric method for building complex models from range images. In: Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, pp. 303–312 (1996)
21.
Zurück zum Zitat Zach, C., Pock, T., Bischof, H.: A globally optimal algorithm for robust TV-L1 range image integration. In: ICCV. IEEE, pp. 1–8 (2007) Zach, C., Pock, T., Bischof, H.: A globally optimal algorithm for robust TV-L1 range image integration. In: ICCV. IEEE, pp. 1–8 (2007)
22.
Zurück zum Zitat Lorensen, W.E., Cline, H.E.: Marching cubes: a high resolution 3D surface construction algorithm. ACM SIGGRAPH Comput. Graph. 21, 163–169 (1987)CrossRef Lorensen, W.E., Cline, H.E.: Marching cubes: a high resolution 3D surface construction algorithm. ACM SIGGRAPH Comput. Graph. 21, 163–169 (1987)CrossRef
23.
Zurück zum Zitat Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)CrossRef Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)CrossRef
24.
Zurück zum Zitat Billings, S., Taylor, R.: Generalized iterative most likely oriented-point (G-IMLOP) registration. IJCARS 10(8), 1213–1226 (2015) Billings, S., Taylor, R.: Generalized iterative most likely oriented-point (G-IMLOP) registration. IJCARS 10(8), 1213–1226 (2015)
25.
Zurück zum Zitat Bernardini, F., Mittleman, J., Rushmeier, H., Silva, C., Taubin, G.: The ball-pivoting algorithm for surface reconstruction. IEEE Trans. Vis. Comput. Graph. 5(4), 349–359 (1999)CrossRef Bernardini, F., Mittleman, J., Rushmeier, H., Silva, C., Taubin, G.: The ball-pivoting algorithm for surface reconstruction. IEEE Trans. Vis. Comput. Graph. 5(4), 349–359 (1999)CrossRef
27.
Zurück zum Zitat Wang, R., Pizer, S.M., Frahm, J.M.: Recurrent neural network for (un-) supervised learning of monocular video visual odometry and depth. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5555–5564 (2019) Wang, R., Pizer, S.M., Frahm, J.M.: Recurrent neural network for (un-) supervised learning of monocular video visual odometry and depth. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5555–5564 (2019)
28.
Zurück zum Zitat Khan, I.: Robust sparse and dense nonrigid structure from motion. IEEE Trans. Multimed. 20(4), 841–850 (2018)CrossRef Khan, I.: Robust sparse and dense nonrigid structure from motion. IEEE Trans. Multimed. 20(4), 841–850 (2018)CrossRef
Metadaten
Titel
Reconstructing Sinus Anatomy from Endoscopic Video – Towards a Radiation-Free Approach for Quantitative Longitudinal Assessment
verfasst von
Xingtong Liu
Maia Stiber
Jindan Huang
Masaru Ishii
Gregory D. Hager
Russell H. Taylor
Mathias Unberath
Copyright-Jahr
2020
DOI
https://doi.org/10.1007/978-3-030-59716-0_1

Premium Partner