Skip to main content
Top

2019 | OriginalPaper | Chapter

Spatial and Spectral Calibration of a Multispectral-Augmented Endoscopic Prototype

Authors : Omar Zenteno, Alexandre Krebs, Sylvie Treuillet, Yves Lucas, Yannick Benezeth, Franck Marzani

Published in: Computer Vision, Imaging and Computer Graphics Theory and Applications

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

We developed a multispectral-augmented endoscopic prototype which increases the common number of bands under analysis, allowing exploration in the visible and near infrared range (400–1000 nm). The prototype combines endoscopy with spectroscopy using white light (WL) or Narrow Band Imaging light (NBI) endoscope and two multispectral cameras connected to a twin-cam splitter. The splitter is then connected to an optical fiber and introduced in the endoscope instrument channel.
In this work, we introduce a spectral calibration and an axial displacement correction function to register both modalities. The former is based on a multi-linear transformation of multispectral bands and its performance is assessed using a Digital SG ColorChecker® pattern to report an RMSE of 6.78%. The latter relates the insertion depth of the fiberscope with the required geometric transformation. The performance was assessed using a chessboard pattern and its corner coordinates as ground truth. The mean RMSE error for the registration using our method was 2.3 ± 0.7 pixels, whereas the RMSE error using a frame by frame homographic registration was 1.2 ± 0.4 pixels. Finally, the technique was tested on mouth exploration samples to simulate in-vivo acquisition and display complete spectra for single points of analysis. The results reveal that our method provides similar performance when compared to a homographic transformation which would be impossible to perform in-vivo.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Bouguet, J.Y.: Matlab camera calibration toolbox. Caltech Technical report (2000) Bouguet, J.Y.: Matlab camera calibration toolbox. Caltech Technical report (2000)
2.
go back to reference Burgers, K., Fessehatsion, Y., Rahmani, S., Seo, J., Wittman, T.: A comparative analysis of dimension reduction algorithms on hyperspectral data. LAMDA Research Group, August 2009 Burgers, K., Fessehatsion, Y., Rahmani, S., Seo, J., Wittman, T.: A comparative analysis of dimension reduction algorithms on hyperspectral data. LAMDA Research Group, August 2009
3.
go back to reference Burschka, D., Li, M., Ishii, M., Taylor, R.H., Hager, G.D.: Scale-invariant registration of monocular endoscopic images to ct-scans for sinus surgery. Med. Image Anal. 9(5), 413–426 (2005)CrossRef Burschka, D., Li, M., Ishii, M., Taylor, R.H., Hager, G.D.: Scale-invariant registration of monocular endoscopic images to ct-scans for sinus surgery. Med. Image Anal. 9(5), 413–426 (2005)CrossRef
4.
go back to reference Clancy, N.T., et al.: Multispectral image alignment using a three channel endoscope in vivo during minimally invasive surgery. Biomed. Opt. Express 3(10), 2567–2578 (2012)CrossRef Clancy, N.T., et al.: Multispectral image alignment using a three channel endoscope in vivo during minimally invasive surgery. Biomed. Opt. Express 3(10), 2567–2578 (2012)CrossRef
5.
go back to reference Deligianni, F.: Visual augmentation for virtual enviromnents in surgical training. Ph.D. thesis, Imperial College London (2006) Deligianni, F.: Visual augmentation for virtual enviromnents in surgical training. Ph.D. thesis, Imperial College London (2006)
6.
go back to reference Dohi, T., Matsumoto, K., Shimoyama, I.: The micro fabry-perot interferometer for the spectral endoscope. In: 18th IEEE International Conference on Micro Electro Mechanical Systems, MEMS 2005, pp. 830–833. IEEE (2005) Dohi, T., Matsumoto, K., Shimoyama, I.: The micro fabry-perot interferometer for the spectral endoscope. In: 18th IEEE International Conference on Micro Electro Mechanical Systems, MEMS 2005, pp. 830–833. IEEE (2005)
7.
go back to reference Galeano, J., Jolivot, R., Benezeth, Y., Marzani, F., Emile, J.F., Lamarque, D.: Analysis of multispectral images of excised colon tissue samples based on genetic algorithms. In: 2012 Eighth International Conference on Signal Image Technology and Internet Based Systems (SITIS), pp. 833–838. IEEE (2012) Galeano, J., Jolivot, R., Benezeth, Y., Marzani, F., Emile, J.F., Lamarque, D.: Analysis of multispectral images of excised colon tissue samples based on genetic algorithms. In: 2012 Eighth International Conference on Signal Image Technology and Internet Based Systems (SITIS), pp. 833–838. IEEE (2012)
8.
go back to reference Georgieva, V.: Homomorphic filtering approach for narrowband images enhancement. J. Appl. Electromagn. (JAE), in print (2015) Georgieva, V.: Homomorphic filtering approach for narrowband images enhancement. J. Appl. Electromagn. (JAE), in print (2015)
9.
go back to reference Hamamoto, Y., Endo, T., Nosho, K., Arimura, Y., Sato, M., Imai, K.: Usefulness of narrow-band imaging endoscopy for diagnosis of Barrett’s esophagus. J. Gastroenterol. 39(1), 14–20 (2004)CrossRef Hamamoto, Y., Endo, T., Nosho, K., Arimura, Y., Sato, M., Imai, K.: Usefulness of narrow-band imaging endoscopy for diagnosis of Barrett’s esophagus. J. Gastroenterol. 39(1), 14–20 (2004)CrossRef
10.
go back to reference Heikkila, J., Silven, O.: A four-step camera calibration procedure with implicit image correction. In: Proceedings of the 1997 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1997, pp. 1106–1112. IEEE (1997) Heikkila, J., Silven, O.: A four-step camera calibration procedure with implicit image correction. In: Proceedings of the 1997 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1997, pp. 1106–1112. IEEE (1997)
11.
go back to reference Kiyotoki, S., et al.: New method for detection of gastric cancer by hyperspectral imaging: a pilot study. J. Biomed. Opt. 18(2), 026010–026010 (2013)CrossRef Kiyotoki, S., et al.: New method for detection of gastric cancer by hyperspectral imaging: a pilot study. J. Biomed. Opt. 18(2), 026010–026010 (2013)CrossRef
12.
go back to reference Koppel, D., Wang, Y.F., Lee, H.: Image-based rendering and modeling in video-endoscopy. In: IEEE International Symposium on Biomedical Imaging: Nano to Macro, pp. 269–272. IEEE (2004) Koppel, D., Wang, Y.F., Lee, H.: Image-based rendering and modeling in video-endoscopy. In: IEEE International Symposium on Biomedical Imaging: Nano to Macro, pp. 269–272. IEEE (2004)
13.
go back to reference Lu, G., Fei, B.: Medical hyperspectral imaging: a review. J. Biomed. Opt. 19(1), 010901 (2014)CrossRef Lu, G., Fei, B.: Medical hyperspectral imaging: a review. J. Biomed. Opt. 19(1), 010901 (2014)CrossRef
14.
go back to reference Martin, M.E., et al.: Development of an advanced hyperspectral imaging (HSI) system with applications for cancer detection. Ann. Biomed. Eng. 34(6), 1061–1068 (2006)CrossRef Martin, M.E., et al.: Development of an advanced hyperspectral imaging (HSI) system with applications for cancer detection. Ann. Biomed. Eng. 34(6), 1061–1068 (2006)CrossRef
15.
go back to reference Martin, R., Thies, B., Gerstner, A.O.: Hyperspectral hybrid method classification for detecting altered mucosa of the human larynx. Int. J. Health Geogr. 11(1), 21 (2012)CrossRef Martin, R., Thies, B., Gerstner, A.O.: Hyperspectral hybrid method classification for detecting altered mucosa of the human larynx. Int. J. Health Geogr. 11(1), 21 (2012)CrossRef
16.
go back to reference Martinez-Herrera, S.E., et al.: Identification of precancerous lesions by multispectral gastroendoscopy. Signal Image Video Process. 10(3), 455–462 (2016)CrossRef Martinez-Herrera, S.E., et al.: Identification of precancerous lesions by multispectral gastroendoscopy. Signal Image Video Process. 10(3), 455–462 (2016)CrossRef
17.
go back to reference Nair, J.J., Govindan, V.: Intensity inhomogeneity correction using modified homomorphic unsharp masking. J. Med. Imaging Health Inform. 4(2), 285–290 (2014)CrossRef Nair, J.J., Govindan, V.: Intensity inhomogeneity correction using modified homomorphic unsharp masking. J. Med. Imaging Health Inform. 4(2), 285–290 (2014)CrossRef
19.
go back to reference Scholz, M., et al.: Development of an endoscopic navigation system based on digital image processing. Comput. Aided Surg. 3(3), 134–143 (1998)CrossRef Scholz, M., et al.: Development of an endoscopic navigation system based on digital image processing. Comput. Aided Surg. 3(3), 134–143 (1998)CrossRef
20.
go back to reference Sharma, G., Bala, R.: Digital Color Imaging Handbook. CRC Press, Boca Raton (2002) Sharma, G., Bala, R.: Digital Color Imaging Handbook. CRC Press, Boca Raton (2002)
21.
go back to reference Song, L.M.W.K., et al.: Narrow band imaging and multiband imaging. Gastrointest. Endosc. 67(4), 581–589 (2008)CrossRef Song, L.M.W.K., et al.: Narrow band imaging and multiband imaging. Gastrointest. Endosc. 67(4), 581–589 (2008)CrossRef
22.
go back to reference Van Der Maaten, L., Postma, E., Van den Herik, J.: Dimensionality reduction: a comparative. J. Mach. Learn. Res. 10, 66–71 (2009) Van Der Maaten, L., Postma, E., Van den Herik, J.: Dimensionality reduction: a comparative. J. Mach. Learn. Res. 10, 66–71 (2009)
23.
go back to reference Westwood, J., et al.: Reconstruction and enhancement in monocular laparoscopic imagery. Medicine Meets Virtual Reality 12: Building a Better You: the Next Tools for Medical Education, Diagnosis, and Care, vol. 98, p. 37 (2004) Westwood, J., et al.: Reconstruction and enhancement in monocular laparoscopic imagery. Medicine Meets Virtual Reality 12: Building a Better You: the Next Tools for Medical Education, Diagnosis, and Care, vol. 98, p. 37 (2004)
24.
go back to reference Zenteno, O., Krebs, A., Treuillet, S., Lucas, Y., Benezeth, Y., Marzani, F.: Dual-channel geometric registration of a multispectral-augmented endoscopic prototype. In: Proceedings of the 13th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 4: VISAPP, pp. 75–82. INSTICC, SciTePress (2018). https://doi.org/10.5220/0006721200750082 Zenteno, O., Krebs, A., Treuillet, S., Lucas, Y., Benezeth, Y., Marzani, F.: Dual-channel geometric registration of a multispectral-augmented endoscopic prototype. In: Proceedings of the 13th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 4: VISAPP, pp. 75–82. INSTICC, SciTePress (2018). https://​doi.​org/​10.​5220/​0006721200750082​
25.
go back to reference Zhang, Z.: Flexible camera calibration by viewing a plane from unknown orientations. In: Proceedings of the Seventh IEEE International Conference on Computer Vision, vol. 1, pp. 666–673. IEEE (1999) Zhang, Z.: Flexible camera calibration by viewing a plane from unknown orientations. In: Proceedings of the Seventh IEEE International Conference on Computer Vision, vol. 1, pp. 666–673. IEEE (1999)
Metadata
Title
Spatial and Spectral Calibration of a Multispectral-Augmented Endoscopic Prototype
Authors
Omar Zenteno
Alexandre Krebs
Sylvie Treuillet
Yves Lucas
Yannick Benezeth
Franck Marzani
Copyright Year
2019
DOI
https://doi.org/10.1007/978-3-030-26756-8_13

Premium Partner