Skip to main content
Top
Published in: International Journal of Computer Assisted Radiology and Surgery 12/2020

16-09-2020 | Original Article

Robust endocytoscopic image classification based on higher-order symmetric tensor analysis and multi-scale topological statistics

Authors: Hayato Itoh, Yukitaka Nimura, Yuichi Mori, Masashi Misawa, Shin-Ei Kudo, Kinichi Hotta, Kazuo Ohtsuka, Shoichi Saito, Yutaka Saito, Hiroaki Ikematsu, Yuichiro Hayashi, Masahiro Oda, Kensaku Mori

Published in: International Journal of Computer Assisted Radiology and Surgery | Issue 12/2020

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Purpose

An endocytoscope is a new type of endoscope that enables users to perform conventional endoscopic observation and ultramagnified observation at the cell level. Although endocytoscopy is expected to improve the cost-effectiveness of colonoscopy, endocytoscopic image diagnosis requires much knowledge and high-level experience for physicians. To circumvent this difficulty, we developed a robust endocytoscopic (EC) image classification method for the construction of a computer-aided diagnosis (CAD) system, since real-time CAD can resolve accuracy issues and reduce interobserver variability.

Method

We propose a novel feature extraction method by introducing higher-order symmetric tensor analysis to the computation of multi-scale topological statistics on an image, and we integrate this feature extraction with EC image classification. We experimentally evaluate the classification accuracy of our proposed method by comparing it with three deep learning methods. We conducted this comparison by using our large-scale multi-hospital dataset of about 55,000 images of over 3800 patients.

Results

Our proposed method achieved an average 90% classification accuracy for all the images in four hospitals even though the best deep learning method achieved 95% classification accuracy for images in only one hospital. In the case with a rejection option, the proposed method achieved expert-level accurate classification. These results demonstrate the robustness of our proposed method against pit pattern variations, including differences of colours, contrasts, shapes, and hospitals.

Conclusions

We developed a robust EC image classification method with novel feature extraction. This method is useful for the construction of a practical CAD system, since it has sufficient generalisation ability.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Mori Y, Kudo SE, Ikehara N, Wada Y, Kutsukawa M, Misawa M, Kudo T, Kobayashi Y, Miyachi H, Yamamura F, Ohtsuka K, Inoue H, Hamatani S (2013) Comprehensive diagnostic ability of endocytoscopy compared with biopsy for colorectal neoplasms: a prospective randomized noninferiority trial. Endoscopy 45(2):98–105CrossRef Mori Y, Kudo SE, Ikehara N, Wada Y, Kutsukawa M, Misawa M, Kudo T, Kobayashi Y, Miyachi H, Yamamura F, Ohtsuka K, Inoue H, Hamatani S (2013) Comprehensive diagnostic ability of endocytoscopy compared with biopsy for colorectal neoplasms: a prospective randomized noninferiority trial. Endoscopy 45(2):98–105CrossRef
2.
go back to reference Mori Y, Kudo SE, Chiu PW, Singh R, Misawa M, Wakamura K, Kudo T, Hayashi T, katagiri A, Miyachi H, Ishida F, Maeda Y, Inoue H, Nimura Y, Oda M, Mori K (2016) Impact of an automated system for endocytoscopic diagnosis of small colorectal lesions: an international web-based study. Endoscopy 48(12):1110–1118CrossRef Mori Y, Kudo SE, Chiu PW, Singh R, Misawa M, Wakamura K, Kudo T, Hayashi T, katagiri A, Miyachi H, Ishida F, Maeda Y, Inoue H, Nimura Y, Oda M, Mori K (2016) Impact of an automated system for endocytoscopic diagnosis of small colorectal lesions: an international web-based study. Endoscopy 48(12):1110–1118CrossRef
3.
go back to reference Fu JJ, Yu YW, Lin HM, Chai JW, Chen CC (2014) Feature extraction and pattern classification of colorectal polyps in colonoscopic imaging. Comput Med Imag Graph 38(4):267–275CrossRef Fu JJ, Yu YW, Lin HM, Chai JW, Chen CC (2014) Feature extraction and pattern classification of colorectal polyps in colonoscopic imaging. Comput Med Imag Graph 38(4):267–275CrossRef
4.
go back to reference Mesejo P, Pizarro D, Abergel A, Rouquette O, Beorchia S, Poincloux L, Bartoli A (2016) Computer-aided classification of gastrointestinal lesions in regular colonoscopy. IEEE Trans Med Imag 35(9):2051–2063CrossRef Mesejo P, Pizarro D, Abergel A, Rouquette O, Beorchia S, Poincloux L, Bartoli A (2016) Computer-aided classification of gastrointestinal lesions in regular colonoscopy. IEEE Trans Med Imag 35(9):2051–2063CrossRef
5.
go back to reference Häfner M, Liedlgruber M, Uhl A, Vécsei A, Wrba F (2012) Color treatment in endoscopic image classification using multi-scale local color vector patterns. Med Imag Anal 16(1):75–86CrossRef Häfner M, Liedlgruber M, Uhl A, Vécsei A, Wrba F (2012) Color treatment in endoscopic image classification using multi-scale local color vector patterns. Med Imag Anal 16(1):75–86CrossRef
6.
go back to reference Tamaki T, Yoshimura J, Kawakami M, Raytchev B, Kaneda K, Yoshida S, Takemura Y, Onji K, Miyaki R, Tanaka S (2013) Computer-aided colorectal tumor classification in NBI endoscopy using local features. Med Imag Anal 17(1):78–100CrossRef Tamaki T, Yoshimura J, Kawakami M, Raytchev B, Kaneda K, Yoshida S, Takemura Y, Onji K, Miyaki R, Tanaka S (2013) Computer-aided colorectal tumor classification in NBI endoscopy using local features. Med Imag Anal 17(1):78–100CrossRef
7.
go back to reference Wimmer G, Tamaki T, Tischendorf JJ, Häfner M, Yoshida S, Tanaka S, Uhl A (2016) Directional wavelet based features for colonic polyp classification. Med Imag Anal 31:16–36CrossRef Wimmer G, Tamaki T, Tischendorf JJ, Häfner M, Yoshida S, Tanaka S, Uhl A (2016) Directional wavelet based features for colonic polyp classification. Med Imag Anal 31:16–36CrossRef
8.
go back to reference Tamaki T, Sonoyama S, Hirakawa T, Raytchev B, Kaneda K, Koide T, Yoshida S, Mieno Hiroshi, Tanaka S (2016) Computer-aided colorectal tumor classification in NBI endoscopy using CNN features. In: Proc Korea–Japan joint workshop on Frontiers of computer vision, pp 61–65 Tamaki T, Sonoyama S, Hirakawa T, Raytchev B, Kaneda K, Koide T, Yoshida S, Mieno Hiroshi, Tanaka S (2016) Computer-aided colorectal tumor classification in NBI endoscopy using CNN features. In: Proc Korea–Japan joint workshop on Frontiers of computer vision, pp 61–65
9.
go back to reference Itoh H, Mori Y, Misawa M, Oda M, Kudo SE, Mori K (2018) Cascade classification of endocytoscopic images of colorectal lesions for automated pathological diagnosis. Proc SPIE Med Imag 10575:269–274 Itoh H, Mori Y, Misawa M, Oda M, Kudo SE, Mori K (2018) Cascade classification of endocytoscopic images of colorectal lesions for automated pathological diagnosis. Proc SPIE Med Imag 10575:269–274
10.
go back to reference Itoh H, Lu Z, Mori Y, Misawa M, Oda M, Kudo SE, Mori K (2018) Discriminative feature selection by optimal manifold search for neoplastic image recognition. Proc ECCV Workshops 4:534–549 Itoh H, Lu Z, Mori Y, Misawa M, Oda M, Kudo SE, Mori K (2018) Discriminative feature selection by optimal manifold search for neoplastic image recognition. Proc ECCV Workshops 4:534–549
11.
go back to reference Kominami Y, Yoshida S, Tanaka S, Sanomura Y, Hirakawa T, Raytchev B, Tamaki T, Koide T, Kaneda K, Chayama K (2016) Computer-aided diagnosis of colorectal polyp histology by using a real-time image recognition system and narrow-band imaging magnifying colonoscopy. Gastrointest Endosc 83:643–649CrossRef Kominami Y, Yoshida S, Tanaka S, Sanomura Y, Hirakawa T, Raytchev B, Tamaki T, Koide T, Kaneda K, Chayama K (2016) Computer-aided diagnosis of colorectal polyp histology by using a real-time image recognition system and narrow-band imaging magnifying colonoscopy. Gastrointest Endosc 83:643–649CrossRef
12.
go back to reference Schultz T, Weickert J, Seidel HP (2009) A higher-order structure tensor. In: Visualization and processing of tensor fields. Springer, Berlin Schultz T, Weickert J, Seidel HP (2009) A higher-order structure tensor. In: Visualization and processing of tensor fields. Springer, Berlin
13.
go back to reference Schultz T (2011) Topological features in 2D symmetric higher-order tensor fields. Comput Gr Forum 30(3):841–850CrossRef Schultz T (2011) Topological features in 2D symmetric higher-order tensor fields. Comput Gr Forum 30(3):841–850CrossRef
14.
go back to reference Haralick RM, Shanmugam K, Dinstein I (1973) Textural features for image classification. IEEE Trans Syst Man Cybern SMC 3(6):610–621CrossRef Haralick RM, Shanmugam K, Dinstein I (1973) Textural features for image classification. IEEE Trans Syst Man Cybern SMC 3(6):610–621CrossRef
15.
go back to reference Bishop CM (2011) Pattern recognition and machine learning, 2nd edn. Springer, Berlin Bishop CM (2011) Pattern recognition and machine learning, 2nd edn. Springer, Berlin
16.
go back to reference Vapnik VN (1998) Statistical learning theory. Wiley, Hoboken Vapnik VN (1998) Statistical learning theory. Wiley, Hoboken
17.
go back to reference Platt JC (1999) Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. In: Advances in large margin classifier, pp 61–74 Platt JC (1999) Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. In: Advances in large margin classifier, pp 61–74
18.
go back to reference Zeiler MD, Fergus R (2014) Visualizing and understanding convolutional networks. In: Proc ECCV, pp 818–833 Zeiler MD, Fergus R (2014) Visualizing and understanding convolutional networks. In: Proc ECCV, pp 818–833
19.
go back to reference Maas AL, Hannun AY, Ng AY (2013) Rectifier nonlinearities improve neural network acoustic models. Proc Int Conf Mach Learn 30(1):3 Maas AL, Hannun AY, Ng AY (2013) Rectifier nonlinearities improve neural network acoustic models. Proc Int Conf Mach Learn 30(1):3
20.
go back to reference Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. Proc Int Conf Mach Learn 37:448–456 Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. Proc Int Conf Mach Learn 37:448–456
21.
go back to reference Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15:1929–1958 Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15:1929–1958
22.
go back to reference He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proc CVPR, pp 770–778 He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proc CVPR, pp 770–778
23.
go back to reference Veit A, Wilber M, Belongie S (2016) Residual networks behave like ensembles of relatively shallow networks. In: Proc NIPS, pp 550–558 Veit A, Wilber M, Belongie S (2016) Residual networks behave like ensembles of relatively shallow networks. In: Proc NIPS, pp 550–558
24.
go back to reference Wang J, Li X, Ling CX (2018) Pelee: a real-time object detection system on mobile devices. In: Proc NIPS, pp 1963–1972 Wang J, Li X, Ling CX (2018) Pelee: a real-time object detection system on mobile devices. In: Proc NIPS, pp 1963–1972
25.
go back to reference He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers: surpassing human-level performance on ImageNet classification. In: Proc ICCV, pp 1026–1034 He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers: surpassing human-level performance on ImageNet classification. In: Proc ICCV, pp 1026–1034
26.
go back to reference Kingma DP, Ba J (2015) Adam: a method for stochastic optimization. In: Proc ICLR, pp 1–13 Kingma DP, Ba J (2015) Adam: a method for stochastic optimization. In: Proc ICLR, pp 1–13
27.
go back to reference Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M, Berg AC, Li FF (2015) ImageNet large scale visual recognition challenge. Int J Comput Vis 115(3):211–252CrossRef Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M, Berg AC, Li FF (2015) ImageNet large scale visual recognition challenge. Int J Comput Vis 115(3):211–252CrossRef
28.
go back to reference Mori Y, Kudo SE, Misawa M, Saito Y, Ikematsu H, Hotta K, Ohtsuka K, Urushibara F, Kataoka S, Ogawa Y, Maeda Y, Takeda K, Nakamura H, Ichimasa K, Kudo T, Hayashi T, Wakamura K, Ishida F, Inoue H, Itoh H, Oda M, Mori K (2018) Real-time use of artificial intelligence in identification of diminutive polyps during colonoscopy: a prospective study. Ann Int Med 169(6):357–366CrossRef Mori Y, Kudo SE, Misawa M, Saito Y, Ikematsu H, Hotta K, Ohtsuka K, Urushibara F, Kataoka S, Ogawa Y, Maeda Y, Takeda K, Nakamura H, Ichimasa K, Kudo T, Hayashi T, Wakamura K, Ishida F, Inoue H, Itoh H, Oda M, Mori K (2018) Real-time use of artificial intelligence in identification of diminutive polyps during colonoscopy: a prospective study. Ann Int Med 169(6):357–366CrossRef
Metadata
Title
Robust endocytoscopic image classification based on higher-order symmetric tensor analysis and multi-scale topological statistics
Authors
Hayato Itoh
Yukitaka Nimura
Yuichi Mori
Masashi Misawa
Shin-Ei Kudo
Kinichi Hotta
Kazuo Ohtsuka
Shoichi Saito
Yutaka Saito
Hiroaki Ikematsu
Yuichiro Hayashi
Masahiro Oda
Kensaku Mori
Publication date
16-09-2020
Publisher
Springer International Publishing
Published in
International Journal of Computer Assisted Radiology and Surgery / Issue 12/2020
Print ISSN: 1861-6410
Electronic ISSN: 1861-6429
DOI
https://doi.org/10.1007/s11548-020-02255-3

Other articles of this Issue 12/2020

International Journal of Computer Assisted Radiology and Surgery 12/2020 Go to the issue

Premium Partner