Skip to main content
Top
Published in: Neural Processing Letters 3/2019

24-07-2018

\(\hbox {U}^2\hbox {F}^2\hbox {S}^2\): Uncovering Feature-level Similarities for Unsupervised Feature Selection

Authors: Xin Zheng, Yanqing Guo, Jun Guo, Xiangwei Kong

Published in: Neural Processing Letters | Issue 3/2019

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Unsupervised feature selection is a critical technique in processing high dimensional data containing redundant and noisy features. Based on sample-level similarities, conventional algorithms select features that can preserve the local structure of data points. However, the similarities among all dimensions of features, which play important roles in feature selection, are neglected. In this paper, we propose a novel method dubbed \(\hbox {U}^2\hbox {F}^2\hbox {S}^2\) by uncovering these pivotal similarities for unsupervised feature selection. A feature-level similarity uncovering loss function is first presented to preserve the local structure of data points at the feature level. Specially, we propose two schemes to measure the feature-level similarities from different perspectives. Then, a joint framework of feature selection and clustering is developed to capture the underlying cluster information. The objective function is efficiently optimized by our proposed iterative algorithm. Extensive experimental results on six publicly available databases demonstrate that \(\hbox {U}^2\hbox {F}^2\hbox {S}^2\) outperforms the state-of-the-arts.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Wang R, Nie F, Hong R, Chang X, Yang X, Yu W (2017) Fast and orthogonal locality preserving projections for dimensionality reduction. IEEE Trans Image Process 26(10):5019–5030MathSciNetCrossRefMATH Wang R, Nie F, Hong R, Chang X, Yang X, Yu W (2017) Fast and orthogonal locality preserving projections for dimensionality reduction. IEEE Trans Image Process 26(10):5019–5030MathSciNetCrossRefMATH
2.
go back to reference Han D, Kim J (2015) Unsupervised simultaneous orthogonal basis clustering feature selection. In: Proceedings of the IEEE conference on computer vision and pattern recognition. IEEE Press, Boston, pp 5016–5023 Han D, Kim J (2015) Unsupervised simultaneous orthogonal basis clustering feature selection. In: Proceedings of the IEEE conference on computer vision and pattern recognition. IEEE Press, Boston, pp 5016–5023
3.
go back to reference He R, Tan T, Wang L, Zheng WS (2012) \(l_{2, 1}\) regularized correntropy for robust feature selection. In: Proceedings of the IEEE conference on computer vision and pattern recognition. IEEE Press, Providence, pp 2504–2511 He R, Tan T, Wang L, Zheng WS (2012) \(l_{2, 1}\) regularized correntropy for robust feature selection. In: Proceedings of the IEEE conference on computer vision and pattern recognition. IEEE Press, Providence, pp 2504–2511
4.
go back to reference Wang K, He R, Wang L, Wang W, Tan T (2016) Joint feature selection and subspace learning for cross-modal retrieval. IEEE Trans Pattern Anal Mach Intell 38(10):2010–2023CrossRef Wang K, He R, Wang L, Wang W, Tan T (2016) Joint feature selection and subspace learning for cross-modal retrieval. IEEE Trans Pattern Anal Mach Intell 38(10):2010–2023CrossRef
5.
go back to reference Nie F, Zhu W, Li X (2016) Unsupervised feature selection with structured graph optimization. In: Thirtieth AAAI conference on artificial intelligence. AAAI Press, Phoenix, AZ, pp 1302–1308 Nie F, Zhu W, Li X (2016) Unsupervised feature selection with structured graph optimization. In: Thirtieth AAAI conference on artificial intelligence. AAAI Press, Phoenix, AZ, pp 1302–1308
6.
go back to reference Nie F, Xiang S, Jia Y, Zhang C, Yan S (2008) Trace ratio criterion for feature selection. In: Proceedings of the twenty-third AAAI conference on artificial intelligence. AAAI Press, Chicago, IL, pp 671–676 Nie F, Xiang S, Jia Y, Zhang C, Yan S (2008) Trace ratio criterion for feature selection. In: Proceedings of the twenty-third AAAI conference on artificial intelligence. AAAI Press, Chicago, IL, pp 671–676
7.
go back to reference Yang S, Hou C, Nie F, Wu Y (2012) Unsupervised maximum margin feature selection via \(L_{2, 1}\)-norm minimization. Neural Comput Appl 21:1791–1799CrossRef Yang S, Hou C, Nie F, Wu Y (2012) Unsupervised maximum margin feature selection via \(L_{2, 1}\)-norm minimization. Neural Comput Appl 21:1791–1799CrossRef
8.
go back to reference Wang D, Nie F, Huang H (2014) Unsupervised feature selection via unified trace ratio formulation and K-means clustering (TRACK). In: European conference, ECML PKDD, vol 8726. Nancy, France, pp 306–321 Wang D, Nie F, Huang H (2014) Unsupervised feature selection via unified trace ratio formulation and K-means clustering (TRACK). In: European conference, ECML PKDD, vol 8726. Nancy, France, pp 306–321
9.
go back to reference He X, Cai D, Niyogi P (2005) Laplacian score for feature selection. In: Conference on neural information processing systems. NIPS, British Columbia, p 189 He X, Cai D, Niyogi P (2005) Laplacian score for feature selection. In: Conference on neural information processing systems. NIPS, British Columbia, p 189
10.
go back to reference Cai D, Zhang C, He X (2010) Unsupervised feature selection for multi-cluster data. In: Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM Press, Washington, pp 333–342 Cai D, Zhang C, He X (2010) Unsupervised feature selection for multi-cluster data. In: Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM Press, Washington, pp 333–342
11.
go back to reference Yang Y, Shen HT, Ma Z, Huang Z, Zhou X (2011) \(l_{2, 1}\)-norm regularized discriminative feature selection for unsupervised learning. In: 21st International joint conference on artificial intelligence. AAAI Press, Barcelona, p 1589 Yang Y, Shen HT, Ma Z, Huang Z, Zhou X (2011) \(l_{2, 1}\)-norm regularized discriminative feature selection for unsupervised learning. In: 21st International joint conference on artificial intelligence. AAAI Press, Barcelona, p 1589
12.
go back to reference Li Z, Yang Y, Liu J, Zhou X, Lu H (2012) Unsupervised feature selection using nonnegative spectral analysis. In: 26th Conference on artificial intelligence. AAAI Press, Toronto, pp 1026–1032 Li Z, Yang Y, Liu J, Zhou X, Lu H (2012) Unsupervised feature selection using nonnegative spectral analysis. In: 26th Conference on artificial intelligence. AAAI Press, Toronto, pp 1026–1032
13.
go back to reference Qian M, Zhai C (2013) Robust unsupervised feature selection. In: 23rd International joint conference on artificial intelligence. AAAI Press, Beijing, pp 1621–1627 Qian M, Zhai C (2013) Robust unsupervised feature selection. In: 23rd International joint conference on artificial intelligence. AAAI Press, Beijing, pp 1621–1627
14.
go back to reference Lee DD, Seung HS (1999) Learning the parts of objects by non-negative matrix factorization. Nature 401:788–791CrossRefMATH Lee DD, Seung HS (1999) Learning the parts of objects by non-negative matrix factorization. Nature 401:788–791CrossRefMATH
15.
go back to reference Ng AY, Jordan MI, Weiss Y (2001) On spectral clustering: analysis and an algorithm. In: Conference on neural information processing systems. NIPS, British Columbia, pp 849–856 Ng AY, Jordan MI, Weiss Y (2001) On spectral clustering: analysis and an algorithm. In: Conference on neural information processing systems. NIPS, British Columbia, pp 849–856
16.
go back to reference Stella XY, Shi J (2003) Multiclass spectral clustering. In: 9th IEEE international conference on computer vision. IEEE Press, Nice, pp 313–319 Stella XY, Shi J (2003) Multiclass spectral clustering. In: 9th IEEE international conference on computer vision. IEEE Press, Nice, pp 313–319
17.
go back to reference Shi J, Malik J (2000) Normalized cuts and image segmentation. IEEE Trans Pattern Anal Mach Intell 22:888–905CrossRef Shi J, Malik J (2000) Normalized cuts and image segmentation. IEEE Trans Pattern Anal Mach Intell 22:888–905CrossRef
19.
go back to reference Nie F, Ding C, Luo D, Huang H (2010) Improved MinMax cut graph clustering with nonnegative relaxation. In: European conference on machine learning and knowledge discovery in databases, vol 6322. Springer, pp 451–466 Nie F, Ding C, Luo D, Huang H (2010) Improved MinMax cut graph clustering with nonnegative relaxation. In: European conference on machine learning and knowledge discovery in databases, vol 6322. Springer, pp 451–466
20.
go back to reference Viklands T (2006) Algorithms for the weighted orthogonal procrustes problem and other least squares problems. Doctoral dissertation, Datavetenskap Viklands T (2006) Algorithms for the weighted orthogonal procrustes problem and other least squares problems. Doctoral dissertation, Datavetenskap
21.
go back to reference Lee DD, Seung HS (2001) Algorithms for non-negative matrix factorization. In: Conference on neural information processing systems. NIPS, British Columbia, pp 556–562 Lee DD, Seung HS (2001) Algorithms for non-negative matrix factorization. In: Conference on neural information processing systems. NIPS, British Columbia, pp 556–562
22.
go back to reference Nie F, Huang H, Cai X, Ding CH (2010) Efficient and robust feature selection via joint \(l_{2, 1}\)-norms minimization. In: Conference on neural information processing systems. NIPS, British Columbia, pp 1813–1821 Nie F, Huang H, Cai X, Ding CH (2010) Efficient and robust feature selection via joint \(l_{2, 1}\)-norms minimization. In: Conference on neural information processing systems. NIPS, British Columbia, pp 1813–1821
23.
go back to reference Nene SA, Nayar SK, Murase H (1996) Columbia object image library (COIL-20). Technical report, CUCS-005-96 Nene SA, Nayar SK, Murase H (1996) Columbia object image library (COIL-20). Technical report, CUCS-005-96
24.
go back to reference Fanty MA, Cole RA (1990) Spoken letter recognition. In: Conference on neural information processing systems. NIPS, Denver, pp 220–226 Fanty MA, Cole RA (1990) Spoken letter recognition. In: Conference on neural information processing systems. NIPS, Denver, pp 220–226
25.
go back to reference Hull JJ (1994) A database for handwritten text recognition research. IEEE Trans Pattern Anal Mach Intell 16:550–554CrossRef Hull JJ (1994) A database for handwritten text recognition research. IEEE Trans Pattern Anal Mach Intell 16:550–554CrossRef
26.
go back to reference Georghiades AS, Belhumeur PN, Kriegman DJ (2001) From few to many: illumination cone models for face recognition under variable lighting and pose. IEEE Trans Pattern Anal Mach Intell 23:643–660CrossRef Georghiades AS, Belhumeur PN, Kriegman DJ (2001) From few to many: illumination cone models for face recognition under variable lighting and pose. IEEE Trans Pattern Anal Mach Intell 23:643–660CrossRef
27.
go back to reference Graham DB, Allinson NM (1998) Characterising virtual eigen signatures for general purpose face recognition. Face Recognit 163:446–456CrossRef Graham DB, Allinson NM (1998) Characterising virtual eigen signatures for general purpose face recognition. Face Recognit 163:446–456CrossRef
28.
go back to reference Samaria FS, Harter AC (1994) Parameterisation of a stochastic model for human face identification. In: 2nd IEEE workshop on applications of computer vision. IEEE Press, Sarasota, pp 138–142 Samaria FS, Harter AC (1994) Parameterisation of a stochastic model for human face identification. In: 2nd IEEE workshop on applications of computer vision. IEEE Press, Sarasota, pp 138–142
Metadata
Title
: Uncovering Feature-level Similarities for Unsupervised Feature Selection
Authors
Xin Zheng
Yanqing Guo
Jun Guo
Xiangwei Kong
Publication date
24-07-2018
Publisher
Springer US
Published in
Neural Processing Letters / Issue 3/2019
Print ISSN: 1370-4621
Electronic ISSN: 1573-773X
DOI
https://doi.org/10.1007/s11063-018-9886-5

Other articles of this Issue 3/2019

Neural Processing Letters 3/2019 Go to the issue