Skip to main content

2019 | OriginalPaper | Buchkapitel

Spatial and Spectral Calibration of a Multispectral-Augmented Endoscopic Prototype

verfasst von : Omar Zenteno, Alexandre Krebs, Sylvie Treuillet, Yves Lucas, Yannick Benezeth, Franck Marzani

Erschienen in: Computer Vision, Imaging and Computer Graphics Theory and Applications

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

We developed a multispectral-augmented endoscopic prototype which increases the common number of bands under analysis, allowing exploration in the visible and near infrared range (400–1000 nm). The prototype combines endoscopy with spectroscopy using white light (WL) or Narrow Band Imaging light (NBI) endoscope and two multispectral cameras connected to a twin-cam splitter. The splitter is then connected to an optical fiber and introduced in the endoscope instrument channel.
In this work, we introduce a spectral calibration and an axial displacement correction function to register both modalities. The former is based on a multi-linear transformation of multispectral bands and its performance is assessed using a Digital SG ColorChecker® pattern to report an RMSE of 6.78%. The latter relates the insertion depth of the fiberscope with the required geometric transformation. The performance was assessed using a chessboard pattern and its corner coordinates as ground truth. The mean RMSE error for the registration using our method was 2.3 ± 0.7 pixels, whereas the RMSE error using a frame by frame homographic registration was 1.2 ± 0.4 pixels. Finally, the technique was tested on mouth exploration samples to simulate in-vivo acquisition and display complete spectra for single points of analysis. The results reveal that our method provides similar performance when compared to a homographic transformation which would be impossible to perform in-vivo.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Bouguet, J.Y.: Matlab camera calibration toolbox. Caltech Technical report (2000) Bouguet, J.Y.: Matlab camera calibration toolbox. Caltech Technical report (2000)
2.
Zurück zum Zitat Burgers, K., Fessehatsion, Y., Rahmani, S., Seo, J., Wittman, T.: A comparative analysis of dimension reduction algorithms on hyperspectral data. LAMDA Research Group, August 2009 Burgers, K., Fessehatsion, Y., Rahmani, S., Seo, J., Wittman, T.: A comparative analysis of dimension reduction algorithms on hyperspectral data. LAMDA Research Group, August 2009
3.
Zurück zum Zitat Burschka, D., Li, M., Ishii, M., Taylor, R.H., Hager, G.D.: Scale-invariant registration of monocular endoscopic images to ct-scans for sinus surgery. Med. Image Anal. 9(5), 413–426 (2005)CrossRef Burschka, D., Li, M., Ishii, M., Taylor, R.H., Hager, G.D.: Scale-invariant registration of monocular endoscopic images to ct-scans for sinus surgery. Med. Image Anal. 9(5), 413–426 (2005)CrossRef
4.
Zurück zum Zitat Clancy, N.T., et al.: Multispectral image alignment using a three channel endoscope in vivo during minimally invasive surgery. Biomed. Opt. Express 3(10), 2567–2578 (2012)CrossRef Clancy, N.T., et al.: Multispectral image alignment using a three channel endoscope in vivo during minimally invasive surgery. Biomed. Opt. Express 3(10), 2567–2578 (2012)CrossRef
5.
Zurück zum Zitat Deligianni, F.: Visual augmentation for virtual enviromnents in surgical training. Ph.D. thesis, Imperial College London (2006) Deligianni, F.: Visual augmentation for virtual enviromnents in surgical training. Ph.D. thesis, Imperial College London (2006)
6.
Zurück zum Zitat Dohi, T., Matsumoto, K., Shimoyama, I.: The micro fabry-perot interferometer for the spectral endoscope. In: 18th IEEE International Conference on Micro Electro Mechanical Systems, MEMS 2005, pp. 830–833. IEEE (2005) Dohi, T., Matsumoto, K., Shimoyama, I.: The micro fabry-perot interferometer for the spectral endoscope. In: 18th IEEE International Conference on Micro Electro Mechanical Systems, MEMS 2005, pp. 830–833. IEEE (2005)
7.
Zurück zum Zitat Galeano, J., Jolivot, R., Benezeth, Y., Marzani, F., Emile, J.F., Lamarque, D.: Analysis of multispectral images of excised colon tissue samples based on genetic algorithms. In: 2012 Eighth International Conference on Signal Image Technology and Internet Based Systems (SITIS), pp. 833–838. IEEE (2012) Galeano, J., Jolivot, R., Benezeth, Y., Marzani, F., Emile, J.F., Lamarque, D.: Analysis of multispectral images of excised colon tissue samples based on genetic algorithms. In: 2012 Eighth International Conference on Signal Image Technology and Internet Based Systems (SITIS), pp. 833–838. IEEE (2012)
8.
Zurück zum Zitat Georgieva, V.: Homomorphic filtering approach for narrowband images enhancement. J. Appl. Electromagn. (JAE), in print (2015) Georgieva, V.: Homomorphic filtering approach for narrowband images enhancement. J. Appl. Electromagn. (JAE), in print (2015)
9.
Zurück zum Zitat Hamamoto, Y., Endo, T., Nosho, K., Arimura, Y., Sato, M., Imai, K.: Usefulness of narrow-band imaging endoscopy for diagnosis of Barrett’s esophagus. J. Gastroenterol. 39(1), 14–20 (2004)CrossRef Hamamoto, Y., Endo, T., Nosho, K., Arimura, Y., Sato, M., Imai, K.: Usefulness of narrow-band imaging endoscopy for diagnosis of Barrett’s esophagus. J. Gastroenterol. 39(1), 14–20 (2004)CrossRef
10.
Zurück zum Zitat Heikkila, J., Silven, O.: A four-step camera calibration procedure with implicit image correction. In: Proceedings of the 1997 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1997, pp. 1106–1112. IEEE (1997) Heikkila, J., Silven, O.: A four-step camera calibration procedure with implicit image correction. In: Proceedings of the 1997 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1997, pp. 1106–1112. IEEE (1997)
11.
Zurück zum Zitat Kiyotoki, S., et al.: New method for detection of gastric cancer by hyperspectral imaging: a pilot study. J. Biomed. Opt. 18(2), 026010–026010 (2013)CrossRef Kiyotoki, S., et al.: New method for detection of gastric cancer by hyperspectral imaging: a pilot study. J. Biomed. Opt. 18(2), 026010–026010 (2013)CrossRef
12.
Zurück zum Zitat Koppel, D., Wang, Y.F., Lee, H.: Image-based rendering and modeling in video-endoscopy. In: IEEE International Symposium on Biomedical Imaging: Nano to Macro, pp. 269–272. IEEE (2004) Koppel, D., Wang, Y.F., Lee, H.: Image-based rendering and modeling in video-endoscopy. In: IEEE International Symposium on Biomedical Imaging: Nano to Macro, pp. 269–272. IEEE (2004)
13.
Zurück zum Zitat Lu, G., Fei, B.: Medical hyperspectral imaging: a review. J. Biomed. Opt. 19(1), 010901 (2014)CrossRef Lu, G., Fei, B.: Medical hyperspectral imaging: a review. J. Biomed. Opt. 19(1), 010901 (2014)CrossRef
14.
Zurück zum Zitat Martin, M.E., et al.: Development of an advanced hyperspectral imaging (HSI) system with applications for cancer detection. Ann. Biomed. Eng. 34(6), 1061–1068 (2006)CrossRef Martin, M.E., et al.: Development of an advanced hyperspectral imaging (HSI) system with applications for cancer detection. Ann. Biomed. Eng. 34(6), 1061–1068 (2006)CrossRef
15.
Zurück zum Zitat Martin, R., Thies, B., Gerstner, A.O.: Hyperspectral hybrid method classification for detecting altered mucosa of the human larynx. Int. J. Health Geogr. 11(1), 21 (2012)CrossRef Martin, R., Thies, B., Gerstner, A.O.: Hyperspectral hybrid method classification for detecting altered mucosa of the human larynx. Int. J. Health Geogr. 11(1), 21 (2012)CrossRef
16.
Zurück zum Zitat Martinez-Herrera, S.E., et al.: Identification of precancerous lesions by multispectral gastroendoscopy. Signal Image Video Process. 10(3), 455–462 (2016)CrossRef Martinez-Herrera, S.E., et al.: Identification of precancerous lesions by multispectral gastroendoscopy. Signal Image Video Process. 10(3), 455–462 (2016)CrossRef
17.
Zurück zum Zitat Nair, J.J., Govindan, V.: Intensity inhomogeneity correction using modified homomorphic unsharp masking. J. Med. Imaging Health Inform. 4(2), 285–290 (2014)CrossRef Nair, J.J., Govindan, V.: Intensity inhomogeneity correction using modified homomorphic unsharp masking. J. Med. Imaging Health Inform. 4(2), 285–290 (2014)CrossRef
19.
Zurück zum Zitat Scholz, M., et al.: Development of an endoscopic navigation system based on digital image processing. Comput. Aided Surg. 3(3), 134–143 (1998)CrossRef Scholz, M., et al.: Development of an endoscopic navigation system based on digital image processing. Comput. Aided Surg. 3(3), 134–143 (1998)CrossRef
20.
Zurück zum Zitat Sharma, G., Bala, R.: Digital Color Imaging Handbook. CRC Press, Boca Raton (2002) Sharma, G., Bala, R.: Digital Color Imaging Handbook. CRC Press, Boca Raton (2002)
21.
Zurück zum Zitat Song, L.M.W.K., et al.: Narrow band imaging and multiband imaging. Gastrointest. Endosc. 67(4), 581–589 (2008)CrossRef Song, L.M.W.K., et al.: Narrow band imaging and multiband imaging. Gastrointest. Endosc. 67(4), 581–589 (2008)CrossRef
22.
Zurück zum Zitat Van Der Maaten, L., Postma, E., Van den Herik, J.: Dimensionality reduction: a comparative. J. Mach. Learn. Res. 10, 66–71 (2009) Van Der Maaten, L., Postma, E., Van den Herik, J.: Dimensionality reduction: a comparative. J. Mach. Learn. Res. 10, 66–71 (2009)
23.
Zurück zum Zitat Westwood, J., et al.: Reconstruction and enhancement in monocular laparoscopic imagery. Medicine Meets Virtual Reality 12: Building a Better You: the Next Tools for Medical Education, Diagnosis, and Care, vol. 98, p. 37 (2004) Westwood, J., et al.: Reconstruction and enhancement in monocular laparoscopic imagery. Medicine Meets Virtual Reality 12: Building a Better You: the Next Tools for Medical Education, Diagnosis, and Care, vol. 98, p. 37 (2004)
24.
Zurück zum Zitat Zenteno, O., Krebs, A., Treuillet, S., Lucas, Y., Benezeth, Y., Marzani, F.: Dual-channel geometric registration of a multispectral-augmented endoscopic prototype. In: Proceedings of the 13th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 4: VISAPP, pp. 75–82. INSTICC, SciTePress (2018). https://doi.org/10.5220/0006721200750082 Zenteno, O., Krebs, A., Treuillet, S., Lucas, Y., Benezeth, Y., Marzani, F.: Dual-channel geometric registration of a multispectral-augmented endoscopic prototype. In: Proceedings of the 13th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 4: VISAPP, pp. 75–82. INSTICC, SciTePress (2018). https://​doi.​org/​10.​5220/​0006721200750082​
25.
Zurück zum Zitat Zhang, Z.: Flexible camera calibration by viewing a plane from unknown orientations. In: Proceedings of the Seventh IEEE International Conference on Computer Vision, vol. 1, pp. 666–673. IEEE (1999) Zhang, Z.: Flexible camera calibration by viewing a plane from unknown orientations. In: Proceedings of the Seventh IEEE International Conference on Computer Vision, vol. 1, pp. 666–673. IEEE (1999)
Metadaten
Titel
Spatial and Spectral Calibration of a Multispectral-Augmented Endoscopic Prototype
verfasst von
Omar Zenteno
Alexandre Krebs
Sylvie Treuillet
Yves Lucas
Yannick Benezeth
Franck Marzani
Copyright-Jahr
2019
DOI
https://doi.org/10.1007/978-3-030-26756-8_13

Premium Partner