Skip to main content
Top
Published in: Knowledge and Information Systems 11/2021

11-10-2021 | Regular Paper

Adaptive weighted least squares regression for subspace clustering

Authors: Noura Bouhlel, Ghada Feki, Chokri Ben Amar

Published in: Knowledge and Information Systems | Issue 11/2021

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

In this research paper, we consider the subspace clustering problem which aims at finding a low-dimensional representation of a high-dimensional data set. In particular, our central focus is upon the least squares regression based on which we elaborate an adaptive weighted least squares regression for subspace clustering. Compared to the least squares regression, we consider the data locality to adaptively select relevant and close samples and discard irrelevant and faraway ones. Additionally, we impose a weight matrix on the representation errors to adaptively highlight the meaningful features and minimize the effect of redundant/noisy ones. Finally, we also add a non-negativity constraint on the representation coefficients to enhance the graph interpretability. These interesting properties allow to build up a more informative and quality graph, thereby yielding very promising clustering results. Extensive experiments on synthetic and real databases demonstrated that our clustering method achieves consistently optimal results, compared to multiple clustering methods.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Footnotes
1
http://www.cad.zju.edu.cn/home/dengcai/Data/FaceData.html.
 
2
http://www.cs.columbia.edu/CAVE/software/softlib.
 
3
http://www.gaussianprocess.org/gpml/data/.
 
4
http://yann.lecun.com/exdb/mnist/.
 
Literature
1.
go back to reference Ben Abdallah F, Feki G, Ben Ammar A, Ben Amar C (2018) Multilevel deep learning-based processing for lifelog image retrieval enhancement. In: 2018 IEEE international conference on systems, man, and cybernetics (SMC), pp 1348–1354 Ben Abdallah F, Feki G, Ben Ammar A, Ben Amar C (2018) Multilevel deep learning-based processing for lifelog image retrieval enhancement. In: 2018 IEEE international conference on systems, man, and cybernetics (SMC), pp 1348–1354
2.
go back to reference Bouchrika T, Zaied M, Jemai O, Ben Amar C (2012) Ordering computers by hand gestures recognition based on wavelet networks. In: CCCA12, pp 1–6 Bouchrika T, Zaied M, Jemai O, Ben Amar C (2012) Ordering computers by hand gestures recognition based on wavelet networks. In: CCCA12, pp 1–6
3.
go back to reference Bouhlel N, Feki G, Ben Ammar A, Ben Amar C (2017) A hypergraph-based reranking model for retrieving diverse social images. In: Lecture Notes in Computer Science, vol 10424 LNCS, Springer International Publishing, Cham, pp 279–291 Bouhlel N, Feki G, Ben Ammar A, Ben Amar C (2017) A hypergraph-based reranking model for retrieving diverse social images. In: Lecture Notes in Computer Science, vol 10424 LNCS, Springer International Publishing, Cham, pp 279–291
4.
go back to reference Bouhlel N, Feki G, Ben Ammar A, Ben Amar C (2020) Hypergraph learning with collaborative representation for image search reranking. Int J Multimedia Inform Retrieval Bouhlel N, Feki G, Ben Ammar A, Ben Amar C (2020) Hypergraph learning with collaborative representation for image search reranking. Int J Multimedia Inform Retrieval
5.
go back to reference Bouhlel N, Ksibi A, Ben Ammar A, Ben Amar C (2016) Semantic-aware framework for Mobile Image Search. In: International conference on intelligent systems design and applications, ISDA, vol 2016, IEEE, pp 479–484 Bouhlel N, Ksibi A, Ben Ammar A, Ben Amar C (2016) Semantic-aware framework for Mobile Image Search. In: International conference on intelligent systems design and applications, ISDA, vol 2016, IEEE, pp 479–484
6.
go back to reference Chen Y, Yi Z (2019) Locality-constrained least squares regression for subspace clustering. Knowledge-Based Systems 163:51–56CrossRef Chen Y, Yi Z (2019) Locality-constrained least squares regression for subspace clustering. Knowledge-Based Systems 163:51–56CrossRef
7.
go back to reference Elhamifar E, Vidal R (2009) Sparse subspace clustering. In: 2009 IEEE conference on computer vision and pattern recognition, pp 2790–2797 Elhamifar E, Vidal R (2009) Sparse subspace clustering. In: 2009 IEEE conference on computer vision and pattern recognition, pp 2790–2797
8.
go back to reference Esser E (2010) Applications of lagrangian-based alternating direction methods and connections to split bregman ernie esser Esser E (2010) Applications of lagrangian-based alternating direction methods and connections to split bregman ernie esser
9.
go back to reference Fakhfakh R, Feki G, Ben Ammar A, Ben Amar C (2016) Personalizing information retrieval: A new model for user preferences elicitation. In: 2016 IEEE international conference on systems, man, and cybernetics (SMC), pp 002091–002096 Fakhfakh R, Feki G, Ben Ammar A, Ben Amar C (2016) Personalizing information retrieval: A new model for user preferences elicitation. In: 2016 IEEE international conference on systems, man, and cybernetics (SMC), pp 002091–002096
10.
go back to reference Feki G, Ben Ammar A, Ben Amar C (2014) Adaptive semantic construction for diversity-based image retrieval. In: Proceedings of the international joint conference on knowledge discovery, knowledge engineering and knowledge management, Vol 1, p 444–449. SCITEPRESS - Science and Technology Publications, Lda, Setubal, PRT Feki G, Ben Ammar A, Ben Amar C (2014) Adaptive semantic construction for diversity-based image retrieval. In: Proceedings of the international joint conference on knowledge discovery, knowledge engineering and knowledge management, Vol 1, p 444–449. SCITEPRESS - Science and Technology Publications, Lda, Setubal, PRT
11.
go back to reference Feki G, Ksibi A, Ben Ammar A, Ben Amar C (2013) Improving image search effectiveness by integrating contextual information Feki G, Ksibi A, Ben Ammar A, Ben Amar C (2013) Improving image search effectiveness by integrating contextual information
12.
go back to reference Georghiades AS, Belhumeur PN, Kriegman DJ (2001) From few to many: illumination cone models for face recognition under variable lighting and pose. IEEE Trans Pattern Anal Mach Intell 23(6):643–660CrossRef Georghiades AS, Belhumeur PN, Kriegman DJ (2001) From few to many: illumination cone models for face recognition under variable lighting and pose. IEEE Trans Pattern Anal Mach Intell 23(6):643–660CrossRef
13.
go back to reference Hoerl AE, Kennard RW (1970) Ridge regression: applications to nonorthogonal problems. Technometrics 12(1):69–82CrossRef Hoerl AE, Kennard RW (1970) Ridge regression: applications to nonorthogonal problems. Technometrics 12(1):69–82CrossRef
14.
go back to reference Hu H, Lin Z, Feng J, Zhou J (2014) Smooth representation clustering. 2014 IEEE conference on computer vision and pattern recognition, pp 3834–3841 Hu H, Lin Z, Feng J, Zhou J (2014) Smooth representation clustering. 2014 IEEE conference on computer vision and pattern recognition, pp 3834–3841
15.
go back to reference Huang J, Nie F, Huang H (2015) A new simplex sparse learning model to measure data similarity for clustering. IJCAI International Joint Conference on Artificial Intelligence 2015-Janua(Ijcai), pp 3569–3575 Huang J, Nie F, Huang H (2015) A new simplex sparse learning model to measure data similarity for clustering. IJCAI International Joint Conference on Artificial Intelligence 2015-Janua(Ijcai), pp 3569–3575
16.
go back to reference Jing L, Ng MK, Huang JZ (2007) An entropy weighting k-means algorithm for subspace clustering of high-dimensional sparse data. IEEE Trans Knowl Data Eng 19(8):1026–1041CrossRef Jing L, Ng MK, Huang JZ (2007) An entropy weighting k-means algorithm for subspace clustering of high-dimensional sparse data. IEEE Trans Knowl Data Eng 19(8):1026–1041CrossRef
17.
go back to reference Ksibi A, Feki G, Ammar AB, Amar C (2013) Effective diversification for ambiguous queries in social image retrieval. In: CAIP Ksibi A, Feki G, Ammar AB, Amar C (2013) Effective diversification for ambiguous queries in social image retrieval. In: CAIP
18.
go back to reference Lai Z, Xu Y, Chen Q, Yang J, Zhang D (2014) Multilinear sparse principal component analysis. IEEE Trans Neural Netw Learn Syst 25(10):1942–1950CrossRef Lai Z, Xu Y, Chen Q, Yang J, Zhang D (2014) Multilinear sparse principal component analysis. IEEE Trans Neural Netw Learn Syst 25(10):1942–1950CrossRef
19.
go back to reference Liu G, Lin Z, Yu Y (2010) Robust Subspace Segmentation by Low-Rank Representation. In: Proceedings of the 27th international conference on international conference on machine learning. Omnipress, Madison, WI, USA Liu G, Lin Z, Yu Y (2010) Robust Subspace Segmentation by Low-Rank Representation. In: Proceedings of the 27th international conference on international conference on machine learning. Omnipress, Madison, WI, USA
20.
go back to reference Liu J, Chen Y, Zhang J, Xu Z (2014) Enhancing Low-Rank Subspace Clustering by Manifold Regularization 23(9): 4022–4030 Liu J, Chen Y, Zhang J, Xu Z (2014) Enhancing Low-Rank Subspace Clustering by Manifold Regularization 23(9): 4022–4030
21.
go back to reference Lu C, Min H, Zhao Z.Q, Zhu L, shuang Huang D, Yan S (2012) Robust and efficient subspace segmentation via least squares regression. In: ECCV (2012) Lu C, Min H, Zhao Z.Q, Zhu L, shuang Huang D, Yan S (2012) Robust and efficient subspace segmentation via least squares regression. In: ECCV (2012)
22.
go back to reference Luxburg UV (2006) A tutorial on spectral clustering. Tech Rep pp 1–32 Luxburg UV (2006) A tutorial on spectral clustering. Tech Rep pp 1–32
23.
go back to reference Nie F, Wang X, Jordan MI, Huang H (2016) The constrained laplacian rank algorithm for graph-based clustering. 30th AAAI Conference on Artificial Intelligence, AAAI 2016 (1), pp 1969–1976 Nie F, Wang X, Jordan MI, Huang H (2016) The constrained laplacian rank algorithm for graph-based clustering. 30th AAAI Conference on Artificial Intelligence, AAAI 2016 (1), pp 1969–1976
24.
go back to reference Peng C, Kang Z, Cheng Q (2017) Subspace clustering via variance regularized ridge regression. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), pp 682–691 Peng C, Kang Z, Cheng Q (2017) Subspace clustering via variance regularized ridge regression. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), pp 682–691
25.
go back to reference Peng X, Yu Z, Yi Z, Tang H (2017) Constructing the L2-graph for robust subspace learning and subspace clustering. IEEE Trans Cybern 47(4):1053–1066CrossRef Peng X, Yu Z, Yi Z, Tang H (2017) Constructing the L2-graph for robust subspace learning and subspace clustering. IEEE Trans Cybern 47(4):1053–1066CrossRef
26.
go back to reference Peng X, Zhang L, Yi Z (2013) Scalable sparse subspace clustering. In: 2013 IEEE conference on computer vision and pattern recognition, pp 430–437 Peng X, Zhang L, Yi Z (2013) Scalable sparse subspace clustering. In: 2013 IEEE conference on computer vision and pattern recognition, pp 430–437
27.
go back to reference Samaria FS, *t, F.S.S., Harter A, Site OA (1994) Parameterisation of a stochastic model for human face identification Samaria FS, *t, F.S.S., Harter A, Site OA (1994) Parameterisation of a stochastic model for human face identification
28.
go back to reference Shi J, Malik J (2000) Normalized cuts and image segmentation. IEEE Trans Pattern Anal Mach Intell 22:888–905CrossRef Shi J, Malik J (2000) Normalized cuts and image segmentation. IEEE Trans Pattern Anal Mach Intell 22:888–905CrossRef
29.
go back to reference Tang M, Nie F, Jain R (2016) Capped lp-norm graph embedding for photo clustering. In: Proceedings of the 24th ACM International Conference on Multimedia, MM ’16, pp 431–435. Association for Computing Machinery, New York, NY, USA Tang M, Nie F, Jain R (2016) Capped lp-norm graph embedding for photo clustering. In: Proceedings of the 24th ACM International Conference on Multimedia, MM ’16, pp 431–435. Association for Computing Machinery, New York, NY, USA
30.
go back to reference Tang M, Nie F, Jain R (2017) A graph regularized dimension reduction method for out-of-sample data. Neurocomputing 225:58–63CrossRef Tang M, Nie F, Jain R (2017) A graph regularized dimension reduction method for out-of-sample data. Neurocomputing 225:58–63CrossRef
32.
go back to reference Wen J, Fang X, Xu Y, Tian C, Fei L (2018) Low-rank representation with adaptive graph regularization. Neural Netw 108:83–96CrossRef Wen J, Fang X, Xu Y, Tian C, Fei L (2018) Low-rank representation with adaptive graph regularization. Neural Netw 108:83–96CrossRef
33.
go back to reference Zheng J, Yang P, Chen S, Shen G, Wang W (2017) Iterative re-constrained group sparse face recognition with adaptive weights learning. IEEE Trans Image Process 26:2408–2423MathSciNetCrossRef Zheng J, Yang P, Chen S, Shen G, Wang W (2017) Iterative re-constrained group sparse face recognition with adaptive weights learning. IEEE Trans Image Process 26:2408–2423MathSciNetCrossRef
34.
go back to reference Zhuang L, Gao H, Lin Z, Ma Y, Zhang X, Yu N (2012) Non-negative low rank and sparse graph for semi-supervised learning. 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2328–2335 Zhuang L, Gao H, Lin Z, Ma Y, Zhang X, Yu N (2012) Non-negative low rank and sparse graph for semi-supervised learning. 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2328–2335
35.
go back to reference Zou H, Hastie TJ (2005) Regularization and variable selection via the elastic net Zou H, Hastie TJ (2005) Regularization and variable selection via the elastic net
Metadata
Title
Adaptive weighted least squares regression for subspace clustering
Authors
Noura Bouhlel
Ghada Feki
Chokri Ben Amar
Publication date
11-10-2021
Publisher
Springer London
Published in
Knowledge and Information Systems / Issue 11/2021
Print ISSN: 0219-1377
Electronic ISSN: 0219-3116
DOI
https://doi.org/10.1007/s10115-021-01612-1

Other articles of this Issue 11/2021

Knowledge and Information Systems 11/2021 Go to the issue

Premium Partner