Skip to main content
Top
Published in: International Journal of Machine Learning and Cybernetics 9/2022

08-03-2022 | Original Article

Unsupervised image clustering algorithm based on contrastive learning and K-nearest neighbors

Authors: Xiuling Zhang, Shuo Wang, Ziyun Wu, Xiaofei Tan

Published in: International Journal of Machine Learning and Cybernetics | Issue 9/2022

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

With the development of the times, people generate a huge amount of data every day, most of which are unlabeled data, but manual labeling needs a lot of time and effort, so unsupervised algorithms are being used more often. This paper proposes an unsupervised image clustering algorithm based on contrastive learning and K-nearest neighbors (CLKNN). CLKNN is trained in two steps, which are the representation learning step and the clustering step. Contrastive learning and K-nearest neighbors have a huge impact on CLKNN. In the representation learning step, firstly CLKNN processes the image by double data augmentation to get two different augmented images; then CLKNN uses double contrastive loss to extract the high-level feature information of the augmented images, maximizing the similarity of row space and maximizing the similarity of column space to ensure the invariance of information. In the clustering step, CLKNN finds the nearest neighbors of each image by K-nearest neighbors, then it maximizes the similarity between each image and its nearest neighbors to get the final result. To test the performance of CLKNN, the experiments are conducted on CIFAR-10, CIFAR-100 and STL-10 in this paper. From the final results, it is clear that CLKNN has better performance than other advanced algorithms.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Show more products
Literature
1.
go back to reference Kang G, Jiang L, Yang Y, Hauptmann AG (2019) Contrastive adaptation network for unsupervised domain adaptation. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp 4888–4897 Kang G, Jiang L, Yang Y, Hauptmann AG (2019) Contrastive adaptation network for unsupervised domain adaptation. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp 4888–4897
2.
go back to reference Caron M, Bojanowski P, Joulin A, Douze M (2018) Deep clustering for unsupervised learning of visual features. Lect Notes Comput Sci 20:139–156CrossRef Caron M, Bojanowski P, Joulin A, Douze M (2018) Deep clustering for unsupervised learning of visual features. Lect Notes Comput Sci 20:139–156CrossRef
3.
go back to reference Zeng S, Zhang B, Zhang Y, Gou J (2020) Dual sparse learning via data augmentation for robust facial image classification. Int J Mach Learn Cybernet 11:1717–1734CrossRef Zeng S, Zhang B, Zhang Y, Gou J (2020) Dual sparse learning via data augmentation for robust facial image classification. Int J Mach Learn Cybernet 11:1717–1734CrossRef
5.
go back to reference Wang W, Song H, Zhao S et al (2019) Learning unsupervised video object segmentation through visual attention. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp 3059–3069 Wang W, Song H, Zhao S et al (2019) Learning unsupervised video object segmentation through visual attention. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp 3059–3069
6.
go back to reference Vo HV, Bach F, Cho M et al (2019) Unsupervised image matching and object discovery as optimization. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp 8279–8288 Vo HV, Bach F, Cho M et al (2019) Unsupervised image matching and object discovery as optimization. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp 8279–8288
7.
go back to reference Fischer A, Igel C (2012) An introduction to restricted Boltzmann machines. Lect Notes Comput Sci 20:14–36CrossRef Fischer A, Igel C (2012) An introduction to restricted Boltzmann machines. Lect Notes Comput Sci 20:14–36CrossRef
8.
go back to reference Rumelhart DE, Hinton GE, Williams RJ (1986) Learning representations by back-propagating errors. Nature 323(6088):533–536CrossRef Rumelhart DE, Hinton GE, Williams RJ (1986) Learning representations by back-propagating errors. Nature 323(6088):533–536CrossRef
9.
go back to reference Lecun Y, Bottou L, Bengio Y et al (1988) Gradient-based learning applied to document recognition. Proc IEEE Int Conf Comput Vis 86(11):533–536 Lecun Y, Bottou L, Bengio Y et al (1988) Gradient-based learning applied to document recognition. Proc IEEE Int Conf Comput Vis 86(11):533–536
10.
go back to reference Dosovitskiy A, Springenberg JT, Riedmiller M, Brox T (2014) Discriminative unsupervised feature learning with convolutional neural networks. Adv Neural Inf Process Syst 20:766–774 Dosovitskiy A, Springenberg JT, Riedmiller M, Brox T (2014) Discriminative unsupervised feature learning with convolutional neural networks. Adv Neural Inf Process Syst 20:766–774
11.
go back to reference Hinton GE, Salakhutdinov RR (2006) Reducing the dimensionality of data with neural networks. Science 313(5786):504–507MathSciNetCrossRef Hinton GE, Salakhutdinov RR (2006) Reducing the dimensionality of data with neural networks. Science 313(5786):504–507MathSciNetCrossRef
12.
go back to reference Masci J, Meier U, Cireşan D et al (2011) Stacked convolutional auto-encoders for hierarchical feature extraction. In: The 21th international conference on artificial neural networks, pp 52–59 Masci J, Meier U, Cireşan D et al (2011) Stacked convolutional auto-encoders for hierarchical feature extraction. In: The 21th international conference on artificial neural networks, pp 52–59
14.
go back to reference Radford A, Metz L, Chintala S (2016) Unsupervised representation learning with deep convolutional generative adversarial networks. arxiv: 1511.06434 Radford A, Metz L, Chintala S (2016) Unsupervised representation learning with deep convolutional generative adversarial networks. arxiv:​ 1511.​06434
15.
go back to reference Xie J, Girshick R, Farhadi A (2016) Unsupervised deep embedding for clustering analysis. In: 33rd international conference on machine learning, vol 1, pp 740–749 Xie J, Girshick R, Farhadi A (2016) Unsupervised deep embedding for clustering analysis. In: 33rd international conference on machine learning, vol 1, pp 740–749
16.
go back to reference Dizaji KG, Herandi A, Deng C et al (2017) Deep clustering via joint convolutional autoencoder embedding and relative entropy minimization. In: Proceedings of the IEEE international conference on computer vision, pp 5747–5756 Dizaji KG, Herandi A, Deng C et al (2017) Deep clustering via joint convolutional autoencoder embedding and relative entropy minimization. In: Proceedings of the IEEE international conference on computer vision, pp 5747–5756
17.
go back to reference Li F, Qiao H, Zhang B (2018) Discriminatively boosted image clustering with fully convolutional auto-encoders. Pattern Recogn 83:161–173CrossRef Li F, Qiao H, Zhang B (2018) Discriminatively boosted image clustering with fully convolutional auto-encoders. Pattern Recogn 83:161–173CrossRef
18.
go back to reference Yang J, Parikh D, Batra D (2016) Joint unsupervised learning of deep representations and image clusters. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 5147–5156 Yang J, Parikh D, Batra D (2016) Joint unsupervised learning of deep representations and image clusters. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 5147–5156
19.
go back to reference Caron M, Bojanowski P, Mairal J et al (2019) Unsupervised pre-training of image features on noncurated data. In: Proceedings of the IEEE international conference on computer vision, pp 2959–2968 Caron M, Bojanowski P, Mairal J et al (2019) Unsupervised pre-training of image features on noncurated data. In: Proceedings of the IEEE international conference on computer vision, pp 2959–2968
20.
go back to reference Huang J, Gong S, Zhu X (2020) Deep semantic clustering by partition confidence maximisation. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp 8846–8855 Huang J, Gong S, Zhu X (2020) Deep semantic clustering by partition confidence maximisation. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp 8846–8855
21.
go back to reference Ji X, Vedaldi A, Henriques J (2019) Invariant information clustering for unsupervised image classification and segmentation. In: Proceedings of the IEEE international conference on computer vision, pp 9864–9873 Ji X, Vedaldi A, Henriques J (2019) Invariant information clustering for unsupervised image classification and segmentation. In: Proceedings of the IEEE international conference on computer vision, pp 9864–9873
22.
go back to reference Hadsell R, Chopra S, LeCun Y (2006) Dimensionality reduction by learning an invariant mapping. Proc IEEE Comput Soc Conf Comput Vis Pattern Recogn 2:1735–1742 Hadsell R, Chopra S, LeCun Y (2006) Dimensionality reduction by learning an invariant mapping. Proc IEEE Comput Soc Conf Comput Vis Pattern Recogn 2:1735–1742
23.
go back to reference Mnih A, Teh YW (2012) A fast and simple algorithm for training neural probabilistic language models. In: Proceedings of the 29th international conference on machine learning, vol 2, pp 1751–1758 Mnih A, Teh YW (2012) A fast and simple algorithm for training neural probabilistic language models. In: Proceedings of the 29th international conference on machine learning, vol 2, pp 1751–1758
25.
go back to reference He K, Fan H, Wu Y, et al (2020) Momentum contrast for unsupervised visual representation learning. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp 9726–9735 He K, Fan H, Wu Y, et al (2020) Momentum contrast for unsupervised visual representation learning. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp 9726–9735
26.
go back to reference Henaff OJ, Srinivas A, De Fauw J et al (2020) Data-efficient image recognition with contrastive predictive coding. In: 37th international conference on machine learning, pp 4130–4140 Henaff OJ, Srinivas A, De Fauw J et al (2020) Data-efficient image recognition with contrastive predictive coding. In: 37th international conference on machine learning, pp 4130–4140
27.
28.
go back to reference Van Gansbeke W, Vandenhende S, Georgoulis S et al (2020) SCAN: learning to classify images without labels. Lect Notes Comput Sci 20:268–285CrossRef Van Gansbeke W, Vandenhende S, Georgoulis S et al (2020) SCAN: learning to classify images without labels. Lect Notes Comput Sci 20:268–285CrossRef
Metadata
Title
Unsupervised image clustering algorithm based on contrastive learning and K-nearest neighbors
Authors
Xiuling Zhang
Shuo Wang
Ziyun Wu
Xiaofei Tan
Publication date
08-03-2022
Publisher
Springer Berlin Heidelberg
Published in
International Journal of Machine Learning and Cybernetics / Issue 9/2022
Print ISSN: 1868-8071
Electronic ISSN: 1868-808X
DOI
https://doi.org/10.1007/s13042-022-01533-7

Other articles of this Issue 9/2022

International Journal of Machine Learning and Cybernetics 9/2022 Go to the issue