Abstract
Environmental microorganism (EM) offers a highly efficient, harmless, and low-cost solution to environmental pollution. They are used in sanitation, monitoring, and decomposition of environmental pollutants. However, this depends on the proper identification of suitable microorganisms. In order to fasten, lower the cost, and increase consistency and accuracy of identification, we propose the novel pairwise deep learning features (PDLFs) to analyze microorganisms. The PDLFs technique combines the capability of handcrafted and deep learning features. In this technique, we leverage the Shi and Tomasi interest points by extracting deep learning features from patches which are centered at interest points’ locations. Then, to increase the number of potential features that have intermediate spatial characteristics between nearby interest points, we use Delaunay triangulation theorem and straight line geometric theorem to pair the nearby deep learning features. The potential of pairwise features is justified on the classification of EMs using SVMs, Linear discriminant analysis, Logistic regression, XGBoost and Random Forest classifier. The pairwise features obtain outstanding results of 99.17%, 91.34%, 91.32%, 91.48%, and 99.56%, which are the increase of about 5.95%, 62.40%, 62.37%, 61.84%, and 3.23% in accuracy, F1-score, recall, precision, and specificity respectively, compared to non-paired deep learning features.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Afzali M, Ghaffari A, Fatemizadeh E, Soltanian-Zadeh H (2016) Medical image registration using sparse coding of image patches. Comput Biol Med 73:56–70
Bengio Y, Courville A, Vincent P (2013) Representation learning: a review and new perspectives. IEEE Trans Pattern Anal Machine Intell 35(8):1798–1828
Bolovinou A, Pratikakis I, Perantonis S (2013) Bag of spatio-visual words for context inference in scene classification. Pattern Recogn 46(3):1039–1053
Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: 2005 IEEE Computer society conference on computer vision and pattern recognition (CVPR’05), vol 1, pp 886–893
Delaunay B, Vide S, Lamémoire A, De Georges V (1934) Bulletin de l’académie des sciences de l’urss. Classe des sciences mathé,matiques et na (6):793–800
Dou J, Li J (2014) Image matching based local delaunay triangulation and affine invariant geometric constraint. Optik 125(1):526–531
Flores M, Torres G, García G, Licona M (2017) Fingerprint verification methods using delaunay triangulations. Int Arab J Inf Technol 14(3):346–354
Gokcen I, Peng J (2002) Comparing linear discriminant analysis and support vector machines. In: International Conference on advances in information systems, pp 104–113
Guo X, Yin Y, Dong C, Yang G, Zhou G (2008) On the class imbalance problem. In: 2008 Fourth international conference on natural computation, vol 4, pp 192–201
Harris C, Stephens M et al (1988) A combined corner and edge detector. In: Alvey vision conference, vol 15, pp 10–5244
Hinchey MG, Sterritt R, Rouff C (2007) Swarms and swarm intelligence. Computer 40 (4):111–113
Keçeli AS, Keçeli SU, Kaya A (2018) Classification of radiolarian fossil images with deep learning methods. In: 2018 26th signal processing and communications applications conference (SIU), pp 1–4
Khan ZN, Qureshi RJ, Ahmad J (2016) On feature based delaunay triangulation for palmprint recognition. arXiv:160201927
Kosov S, Shirahama K, Li C, Grzegorzek M (2018) Environmental microorganism classification using conditional random fields and deep convolutional neural networks. Pattern Recogn 77:248–261
Kruk M, Kozera R, Osowski S, Trzciński P, Paszt LS, Sumorok B, Borkowski B (2015) Computerized classification system for the identification of soil microorganisms. In: AIP conference proceedings. vol 1648, p 660018
Kuzovkin I, Vicente R, Petton M, Lachaux JP, Baciu M, Kahane P, Rheims S, Vidal JR, Aru J (2018) Activations of deep convolutional neural networks are aligned with gamma band activity of human visual cortex. Communications Biology 1(1):1–12
Lazebnik S, Schmid C, Ponce J (2005) A maximum entropy framework for part-based texture and object recognition. In: Tenth IEEE international conference on computer vision (ICCV’05) volume 1, vol 1, pp 832–838
Li C, Shirahama K, Grzegorzek M, Ma F, Zhou B (2013) Classification of environmental microorganisms in microscopic images using shape features and support vector machines. In: 2013 IEEE international conference on image processing, pp 2435–2439
Li C, Wang K, Xu N (2019) A survey for the applications of content-based microscopic image analysis in microorganism classification domains. Artif Intell Rev 51(4):577–646
Li C, Kulwa F, Zhang J, Li Z, Xu H, Zhao X (2020) A review of clustering methods in microorganism image analysis. Information Technology in Biomedicine, 13–25
Liang CM, Lai CC, Wang SH, Lin YH (2021) Environmental microorganism classification using optimized deep learning model. Environ Sci Pollut Res, 1–13
Lindeberg T (2013) Scale selection properties of generalized scale-space interest point detectors. Journal of Mathematical Imaging and Vision 46(2):177–210
Ling H, Soatto S (2007) Proximity distribution kernels for geometric context in category recognition. In: 2007 IEEE 11th international conference on computer vision, pp 1–8
Liu D, Hua G, Viola P, Chen T (2008) Integrated feature selection and higher-order spatial feature extraction for object categorization. In: 2008 IEEE conference on computer vision and pattern recognition, pp 1–8
Luo JY, Irisson JO, Graham B, Guigand C, Sarafraz A, Mader C, Cowen RK (2018) Automated plankton image analysis using convolutional neural networks. Limnol Oceanogr Methods 16(12):814–827
Mithra K, Emmanuel WS (2019) Automated identification of mycobacterium bacillus from sputum images for tuberculosis diagnosis. SIViP 13(8):1585–1592
Morioka N, Satoh S (2010) Building compact local pairwise codebook with joint feature space clustering. In: European conference on computer vision, pp 692–705
Morioka N, Satoh S (2011) Compact correlation coding for visual object categorization. In: 2011 International conference on computer vision, pp 1639–1646
Peng W, Hongling X, Wenlin L, Wenlong S (2016) Harris scale invariant corner detection algorithm based on the significant region. International Journal of Signal Processing Image Processing and Pattern Recognition 9(3):413–420
Priya E, Srinivasan S (2016) Automated object and image level classification of tb images using support vector neural network classifier. Biocybernetics and Biomedical Engineering 36(4):670–678
Rajaraman S, Candemir S, Kim I, Thoma G, Antani S (2018) Visualization and interpretation of convolutional neural network predictions in detecting pneumonia in pediatric chest radiographs. Appl Sci 8(10):1715
Savarese S, Winn J, Criminisi A (2006) Discriminative object class models of appearance and shape by correlatons. In: 2006 IEEE Computer society conference on computer vision and pattern recognition (CVPR’06), vol 2, pp 2033–2040
Schmid C, Mohr R, Bauckhage C (2000) Evaluation of interest point detectors. International Journal of computer vision 37(2):151–172
Shi J et al (1994) Good features to track. In: 1994 Proceedings of IEEE conference on computer vision and pattern recognition, pp 593–600
Shin HC, Roth HR, Gao M, Lu L, Xu Z, Nogues I, Yao J, Mollura D, Summers RM (2016) Deep convolutional neural networks for computer-aided detection: Cnn architectures, dataset characteristics and transfer learning. IEEE Transactions on Medical Imaging 35(5):1285–1298
Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:14091556
Singh RL (2017) Principles and applications of environmental biotechnology for a sustainable future
Statnikov A, Aliferis CF (2007) Are random forests better than support vector machines for microarray-based cancer classification?. In: AMIA Annual symposium proceedings, american medical informatics association, vol 2007, p 686
Statnikov A, Aliferis CF, Tsamardinos I, Hardin D, Levy S (2005) A comprehensive evaluation of multicategory classification methods for microarray gene expression cancer diagnosis. Bioinformatics 21(5):631–643
Wang YH (2014) Research on segmentation of protozoan and metazoan image in microscopic examination of activated sludge. In: Applied Mechanics and Materials, vol 448, pp 367– 370
Yamaguchi T, Kawakami S, Hatamoto M, Imachi H, Takahashi M, Araki N, Yamaguchi T, Kubota K (2015) In situ dna-hybridization chain reaction (hcr): a facilitated in situ hcr system for the detection of environmental microorganisms. Environmental Microbiology 17(7):2532–2541
Yang C, Li C, Tiebe O, Shirahama K, Grzegorzek M (2014) Shape-based classification of environmental microorganisms. In: 2014 22nd international conference on pattern recognition, pp 3374–3379
Zhang J, Li C, Kosov S, Grzegorzek M, Shirahama K, Jiang T, Sun C, Li Z, Li H (2021) LCU-Net: A Novel Low-cost U-Net for Environmental Microorganism Image Segmentation. Pattern Recognition p Online first
Zhang Y, Chen T (2009) Efficient kernels for identifying unbounded-order spatial features. In: 2009 IEEE conference on computer vision and pattern recognition, pp 1762–1769
Zhou Q, Li K, Jun X, Bo L (2009) Role and functions of beneficial microorganisms in sustainable aquaculture. Bioresour Technol 100(16):3780–3786
Zieliński B, Plichta A, Misztal K, Spurek P, Brzychczy-włoch M, Ochońska D (2017) Deep learning approach to bacterial colony classification. PloS one 12(9):e0184554
Zou Y, Li C, Shiriham K, Schmidt F, Jiang T, Grzegorzek M (2016) Content-based microscopic image retrieval of environmental microorganisms using multiple colour channels fusion. In: Computer and information science, pp 119–130
Acknowledgements
We thank Prof. Beihai Zhou, Dr. Fangshu Ma from the University of Science and Technology Beijing, PR China, and Prof. Yanling Zou from Freiburg University, Germany, for their previous cooperation in this work. We thank Miss Zixian Li and Mr. Guoxian Li for their important discussion. We also thank B.E. Xuemin Zhu from Johns Hopkins University, USA, and B.E. Bolin Lu from Huazhong University of Science and Technology, PR China, for their careful work in the EMDS-5 image data preparation.
Funding
This work received financial support from the “Natural Science Foundation of China” (No. 61806047).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no competing interests.
Additional information
Responsible Editor: Philippe Garrigues
Availability of data and materials
The datasets analyzed during this study are available in the NEUZihan/EMDS-5 repository, link: https://github.com/NEUZihan/EMDS-5
Author contribution
Frank Kulwa: conceptualization, investigation, methodology, software, writing original draft; Chen Li: conceptualization, investigation, methodology, software, supervision, resources, writing original draft, proofreading; Jinghua Zhang: methodology, proofreading; Kimiaki Shirahama: methodology, validation, resources; Sergey Kosov: methodology, validation; Tao Jiang: methodology, validation, proofreading; Marcin Grzegorzek: methodology, validation, resources; Xin Zhao: proofreading.
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Kulwa, F., Li, C., Zhang, J. et al. A new pairwise deep learning feature for environmental microorganism image analysis. Environ Sci Pollut Res 29, 51909–51926 (2022). https://doi.org/10.1007/s11356-022-18849-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11356-022-18849-0