Top

International Journal of Computer Assisted Radiology and Surgery

Published in:

04-04-2022 | Original Article

Disparity-constrained stereo endoscopic image super-resolution

Authors: Tianyi Zhang, Yun Gu, Xiaolin Huang, Jie Yang, Guang-Zhong Yang

Published in: International Journal of Computer Assisted Radiology and Surgery | Issue 5/2022

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Purpose

With the increasing usage of stereo cameras in computer-assisted surgery techniques, surgeons can benefit from better 3D context of the surgical site in minimally invasive operations. However, since stereo cameras are placed together at the confined endoscope tip, the size of lens and sensors is limited, resulting in low resolution of stereo endoscopic images. How to effectively exploit and utilize stereo information in stereo endoscopic super-resolution (SR) becomes a challenging problem.

Methods

In this work, we propose a disparity-constrained stereo super-resolution network (DCSSRnet) to reconstruct images using a stereo image pair. In particular, a disparity constraint mechanism is incorporated into the generation of SR images in the deep neural network framework with effective feature extractors and atrous parallax attention modules.

Results

Extensive experiments were conducted to evaluate the performance of proposed DCSSRnet on the da Vinci dataset and Medtronic dataset. The results on endoscopic image datasets demonstrate that the proposed approach produces a more effective improvement over current SR methods on both quantitative measurements. The ablation studies further verify the effectiveness of the components of the proposed framework.

Conclusion

The proposed DCSSRnet provides a promising solution on enhancing the spatial resolution of stereo endoscopic image pairs. Specifically, the disparity consistency of the stereo image pair provides informative supervision for image reconstruction. The proposed model can serve as a tool for improving the quality of stereo endoscopic images of endoscopic surgery systems.

previous article LTSP: long-term slice propagation for accurate airway segmentation

next article Surgical biomicroscopy-guided intra-operative optical coherence tomography (iOCT) image super-resolution

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Shenbagaraj L, Thomas-Gibson S, Stebbing J, Broughton R, Dron M, Johnston D, Shaw T, Haboubi HN, Green JT (2019) Endoscopy in 2017: a national survey of practice in the UK. Front Gastroenterol 10(1):7–15. https://doi.org/10.1136/flgastro-2018-100970CrossRef

Kim M, Lee C, Hong N, Kim YJ, Kim S (2017) Development of stereo endoscope system with its innovative master interface for continuous surgical operation. Biomed Eng Online 16(1):1–16. https://doi.org/10.1186/s12938-017-0376-1CrossRef

Dong C, Loy CC, He K, Tang X (2015) Image super-resolution using deep convolutional networks. IEEE Trans on Pattern Anal Machine Intell 38(2):295–307. https://doi.org/10.1109/TPAMI.2015.2439281CrossRef

Kim J, Lee JK, Lee KM (2016) Accurate image super-resolution using very deep convolutional networks. . In Proceedings of IEEE conference on Computer Vision and Pattern Recognition (CVPR) 1646–1654. https://doi.org/10.1109/CVPR.2016.182

Ahn N, Kang B, Sohn KA (2018) Fast, accurate, and lightweight super-resolution with cascading residual network. In Proceedings of European Conference on Computer Vision (ECCV): 252–268. https://doi.org/10.1007/978-3-030-01249-6_16

Jeon DS, Baek SH, Choi I, Min HK (2018) Enhancing the Spatial Resolution of Stereo Images Using a Parallax Prior. In Proceedings of IEEE conference on Computer Vision and Pattern Recognition (CVPR): 1721–1730. https://doi.org/10.1109/CVPR.2018.00185

Yang J, Wright J, Huang TS, Ma Y (2010) Image super-resolution via sparse representation. IEEE Trans Image Process 19(11):2861–2873. https://doi.org/10.1109/TIP.2010.2050625CrossRefPubMed

Glasner D, Bagon S, Irani M (2009) Super-resolution from a single image. In Proceedings of IEEE international conference on computer vision (ICCV): 349–356. https://doi.org/10.1109/ICCV.2009.5459271

Freeman WT, Jones TR, Pasztor EC (2002) Example-based super-resolution. IEEE Comput Graph Appl 22(2):56–65. https://doi.org/10.1109/38.988747CrossRef

10.

Tai Y, Yang J, Liu X (2017) Image super-resolution via deep recursive residual network. In Proceedings of IEEE conference on Computer Vision and Pattern Recognition (CVPR): 2790–2798. https://doi.org/10.1109/CVPR.2017.298

11.

Wang L, Guo Y, Wang Y, Liang Z, Lin Z, Yang J, An W (2020) Parallax attention for unsupervised stereo correspondence learning. IEEE transactions on pattern analysis and machine intelligence. https://doi.org/10.1109/TPAMI.2020.3026899

12.

Zhang Y, Li K, Li K, Wang L, Zhong B, Fu Y (2018) Image super-resolution using very deep residual channel attention networks. In Proceedings of European conference on computer vision (ECCV): 286–301. https://doi.org/10.1007/978-3-030-01234-2_18

13.

Chen Y, Shi F, Christodoulou A G, Xie Y, Zhou Z, Li D (2018) Efficient and accurate MRI super-resolution using a generative adversarial network and 3D multi-level densely connected network. In Proceedings of International conference on medical image computing and computer-assisted intervention (MICCAI): 91–99. https://doi.org/10.1007/978-3-030-00928-1_11

14.

Zhao C, Carass A, Dewey B E, Woo J, Oh J, Calabresi P A, Reich D S, Sati P, Pham D L, Prince J L (2018). A deep learning based anti-aliasing self super-resolution algorithm for MRI. In Proceedings of International conference on medical image computing and computer-assisted intervention (MICCAI): 100–108. https://doi.org/10.1007/978-3-030-00928-1_12

15.

Szczotka AB, Shakir DI, Ravì D, Clarkson MJ, Pereira SP, Vercauteren T (2020) Learning from irregularly sampled data for endomicroscopy super-resolution: a comparative study of sparse and dense approaches. Int J Comput Assis Radiol Surg 15(7):1167–1175. https://doi.org/10.1007/s11548-020-02170-7CrossRef

16.

Bhavsar AV, Rajagopalan AN (2010) Resolution enhancement in multi-image stereo. IEEE Trans Pattern Anal Machine Intell 32(9):1721–1728. https://doi.org/10.1109/TPAMI.2010.90CrossRef

17.

Yang M, Yu K, Zhang C, Li Z, Yang K (2018) DenseASPP for semantic segmentation in street scenes. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR): 3684–3692. https://doi.org/10.1109/CVPR.2018.00388

18.

Lerotic M, Yang G-Z (2006) The Use of Super Resolution in Robotic Assisted Minimally Invasive Surgery. In Proceedings of International conference on medical image computing and computer-assisted intervention (MICCAI): 462–469. https://doi.org/10.3109/10929080701727777

19.

Wang R, Zhang D, Li Q, Zhou XY, Lo B (2021) Real-time Surgical Environment Enhancement for Robot-Assisted Minimally Invasive Surgery Based on Super-Resolution. In Proceedings of international conference on robotics and automation (ICRA): 3434–3440

Title: Disparity-constrained stereo endoscopic image super-resolution
Authors: Tianyi Zhang
Yun Gu
Xiaolin Huang
Jie Yang
Guang-Zhong Yang
Publication date: 04-04-2022
Publisher: Springer International Publishing
Published in: International Journal of Computer Assisted Radiology and Surgery / Issue 5/2022
Print ISSN: 1861-6410
Electronic ISSN: 1861-6429
DOI: https://doi.org/10.1007/s11548-022-02611-5

Springer Professional

Abstract

Purpose

Methods

Results

Conclusion

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Other articles of this Issue 5/2022

Coarse label refinement for improving prostate cancer detection in ultrasound imaging

IJCARS—IPCAI 2022 special issue: 13th international conference on information processing in computer-assisted interventions 2022—part 1

Large scale simulation of labeled intraoperative scenes in unity

Can uncertainty estimation predict segmentation performance in ultrasound bone imaging?

The impact of visualization paradigms on the detectability of spatial misalignment in mixed reality surgical guidance

PATG: position-aware temporal graph networks for surgical phase recognition on laparoscopic videos

Premium Partner