Skip to main content
Top

2016 | OriginalPaper | Chapter

Unsupervised Hypergraph Feature Selection with Low-Rank and Self-Representation Constraints

Authors : Wei He, Xiaofeng Zhu, Yonggang Li, Rongyao Hu, Yonghua Zhu, Shichao Zhang

Published in: Advanced Data Mining and Applications

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Unsupervised feature selection is designed to select a subset of informative features from unlabeled data to avoid the issue of ‘curse of dimensionality’ and thus achieving efficient calculation and storage. In this paper, we integrate the feature-level self-representation property, a low-rank constraint, a hypergraph regularizer, and a sparsity inducing regularizer (i.e., an \(\ell _{2,1}\)-norm regularizer) in a unified framework to conduct unsupervised feature selection. Specifically, we represent each feature by other features to rank the importance of features via the feature-level self-representation property. We then embed a low-rank constraint to consider the relations among features and a hypergarph regularizer to consider both the high-order relations and the local structure of the samples. We finally use an \(\ell _{2,1}\)-norm regularizer to result in low-sparsity to output informative features which satisfy the above constraints. The resulting feature selection model thus takes into account both the global structure of the samples (via the low-rank constraint) and the local structure of the data (via the hypergraph regularizer), rather than only considering each of them used in the previous studies. This enables the proposed model more robust than the previous models due to achieving the stable feature selection model. Experimental results on benchmark datasets showed that the proposed method effectively selected the most informative features by removing the adverse effect of redundant/nosiy features, compared to the state-of-the-art methods.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Cai, X., Ding, C., Nie, F., Huang, H.: On the equivalent of low-rank linear regressions and linear discriminant analysis based regressions. In: SIGKDD, pp. 1124–1132 (2013) Cai, X., Ding, C., Nie, F., Huang, H.: On the equivalent of low-rank linear regressions and linear discriminant analysis based regressions. In: SIGKDD, pp. 1124–1132 (2013)
2.
go back to reference Cao, J., Wu, Z., Wu, J.: Scaling up cosine interesting pattern discovery: a depth-first method. Inf. Sci. 266(5), 31–46 (2014)CrossRef Cao, J., Wu, Z., Wu, J.: Scaling up cosine interesting pattern discovery: a depth-first method. Inf. Sci. 266(5), 31–46 (2014)CrossRef
3.
go back to reference Cheng, D., Zhang, S., Liu, X., Sun, K., Zong, M.: Feature selection by combining subspace learning with sparse representation. Multimedia Syst., 1–7 (2015) Cheng, D., Zhang, S., Liu, X., Sun, K., Zong, M.: Feature selection by combining subspace learning with sparse representation. Multimedia Syst., 1–7 (2015)
4.
go back to reference Gao, L., Song, J., Nie, F., Yan, Y., Sebe, N., Tao Shen, H.: Optimal graph learning with partial tags and multiple features for image and video annotation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4371–4379 (2015) Gao, L., Song, J., Nie, F., Yan, Y., Sebe, N., Tao Shen, H.: Optimal graph learning with partial tags and multiple features for image and video annotation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4371–4379 (2015)
5.
go back to reference Gao, L.L., Song, J., Shao, J., Zhu, X., Shen, H.T.: Zero-shot image categorization by image correlation exploration. In: ICMR, pp. 487–490 (2015) Gao, L.L., Song, J., Shao, J., Zhu, X., Shen, H.T.: Zero-shot image categorization by image correlation exploration. In: ICMR, pp. 487–490 (2015)
6.
go back to reference Gheyas, I.A., Smith, L.S.: Feature subset selection in large dimensionality domains. Pattern Recogn. 43(1), 5–13 (2010)CrossRefMATH Gheyas, I.A., Smith, L.S.: Feature subset selection in large dimensionality domains. Pattern Recogn. 43(1), 5–13 (2010)CrossRefMATH
7.
go back to reference Gu, Q., Li, Z., Han, J.: Joint feature selection and subspace learning. IJCAI 22, 1294–1299 (2011) Gu, Q., Li, Z., Han, J.: Joint feature selection and subspace learning. IJCAI 22, 1294–1299 (2011)
8.
go back to reference He, X., Cai, D., Niyogi, P.: Laplacian score for feature selection. In: NIPS, pp. 507–514 (2005) He, X., Cai, D., Niyogi, P.: Laplacian score for feature selection. In: NIPS, pp. 507–514 (2005)
9.
go back to reference Hou, C., Nie, F., Li, X., Yi, D., Wu, Y.: Joint embedding learning and sparse regression: a framework for unsupervised feature selection. IEEE Trans. Cybern. 44(6), 793–804 (2013) Hou, C., Nie, F., Li, X., Yi, D., Wu, Y.: Joint embedding learning and sparse regression: a framework for unsupervised feature selection. IEEE Trans. Cybern. 44(6), 793–804 (2013)
10.
go back to reference Hu, R., Zhu, X., Cheng, D., He, W., Yan, Y., Song, J., Zhang, S.: Graph self-representation method for unsupervised feature selection. Neurocomputing (2016) Hu, R., Zhu, X., Cheng, D., He, W., Yan, Y., Song, J., Zhang, S.: Graph self-representation method for unsupervised feature selection. Neurocomputing (2016)
11.
go back to reference Huang, Y., Liu, Q., Lv, F., Gong, Y., Metaxas, D.N.: Unsupervised image categorization by hypergraph partition. IEEE Trans. Pattern Anal. Mach. Intell. 33(6), 1266–1273 (2011)CrossRef Huang, Y., Liu, Q., Lv, F., Gong, Y., Metaxas, D.N.: Unsupervised image categorization by hypergraph partition. IEEE Trans. Pattern Anal. Mach. Intell. 33(6), 1266–1273 (2011)CrossRef
12.
go back to reference Jie, C., Wu, Z., Wu, J., Hui, X.: Sail: summation-based incremental learning for information-theoretic text clustering. ieee trans. syst. man cybern. part b cybern. 43(2), 570–584 (2013). A Publication of the IEEE Systems Man & Cybernetics Society Jie, C., Wu, Z., Wu, J., Hui, X.: Sail: summation-based incremental learning for information-theoretic text clustering. ieee trans. syst. man cybern. part b cybern. 43(2), 570–584 (2013). A Publication of the IEEE Systems Man & Cybernetics Society
13.
go back to reference Lewandowski, M., Makris, D., Velastin, S., Nebel, J.-C.: Structural Laplacian eigenmaps for modeling sets of multivariate sequences. IEEE Trans. Cybern. 44(6), 936–949 (2014)CrossRef Lewandowski, M., Makris, D., Velastin, S., Nebel, J.-C.: Structural Laplacian eigenmaps for modeling sets of multivariate sequences. IEEE Trans. Cybern. 44(6), 936–949 (2014)CrossRef
14.
go back to reference Liu, G., Lin, Z., Yan, S., Sun, J., Yu, Y., Ma, Y.: Robust recovery of subspace structures by low-rank representation. IEEE Trans. Softw. Eng. 35 (2013) Liu, G., Lin, Z., Yan, S., Sun, J., Yu, Y., Ma, Y.: Robust recovery of subspace structures by low-rank representation. IEEE Trans. Softw. Eng. 35 (2013)
15.
go back to reference Liu, G., Lin, Z., Yu, Y.: Robust subspace segmentation by low-rank representation. In: CVPR, pp. 663–670 (2010) Liu, G., Lin, Z., Yu, Y.: Robust subspace segmentation by low-rank representation. In: CVPR, pp. 663–670 (2010)
16.
go back to reference Liu, R., Yang, N., Ding, X., Ma, L.: An unsupervised feature selection algorithm: Laplacian score combined with distance-based entropy measure. In: IITA, pp. 65–68 (2009) Liu, R., Yang, N., Ding, X., Ma, L.: An unsupervised feature selection algorithm: Laplacian score combined with distance-based entropy measure. In: IITA, pp. 65–68 (2009)
17.
go back to reference Maugis, C., Celeux, G., Martin-Magniette, M.L.: Variable selection for clustering with gaussian mixture models. Biometrics 65(3), 701–709 (2009)MathSciNetCrossRefMATH Maugis, C., Celeux, G., Martin-Magniette, M.L.: Variable selection for clustering with gaussian mixture models. Biometrics 65(3), 701–709 (2009)MathSciNetCrossRefMATH
18.
go back to reference Nie, F., Huang, H., Cai, X., Ding, C.H.: Efficient and robust feature selection via joint \(\ell _{2,1}\)-norms minimization. In: NIPS, pp. 1813–1821 (2010) Nie, F., Huang, H., Cai, X., Ding, C.H.: Efficient and robust feature selection via joint \(\ell _{2,1}\)-norms minimization. In: NIPS, pp. 1813–1821 (2010)
19.
go back to reference Nie, F., Xiang, S., Jia, Y., Zhang, C., Yan, S.: Trace ratio criterion for feature selection. In: AAAI, pp. 671–676 (2008) Nie, F., Xiang, S., Jia, Y., Zhang, C., Yan, S.: Trace ratio criterion for feature selection. In: AAAI, pp. 671–676 (2008)
20.
go back to reference Peng, Y., Long, X., Lu, B.L.: Graph based semi-supervised learning via structure preserving low-rank representation. Neural Process. Lett. 41(3), 389–406 (2015)CrossRef Peng, Y., Long, X., Lu, B.L.: Graph based semi-supervised learning via structure preserving low-rank representation. Neural Process. Lett. 41(3), 389–406 (2015)CrossRef
21.
go back to reference Qin, Y., Zhang, S., Zhu, X., Zhang, J., Zhang, C.: Semi-parametric optimization for missing data imputation. Appl. Intell. 27(1), 79–88 (2007)CrossRefMATH Qin, Y., Zhang, S., Zhu, X., Zhang, J., Zhang, C.: Semi-parametric optimization for missing data imputation. Appl. Intell. 27(1), 79–88 (2007)CrossRefMATH
22.
go back to reference Shi, X., Guo, Z., Lai, Z., Yang, Y., Bao, Z., Zhang, D.: A framework of joint graph embedding and sparse regression for dimensionality reduction. IEEE Trans. Image Process. 24(4), 1341–1355 (2015). A Publication of the IEEE Signal Processing SocietyMathSciNetCrossRef Shi, X., Guo, Z., Lai, Z., Yang, Y., Bao, Z., Zhang, D.: A framework of joint graph embedding and sparse regression for dimensionality reduction. IEEE Trans. Image Process. 24(4), 1341–1355 (2015). A Publication of the IEEE Signal Processing SocietyMathSciNetCrossRef
23.
go back to reference Sunzhong, L.V., Jiang, H., Zhao, L., Wang, D., Fan, M.: Manifold based fisher method for semi-supervised feature selection. In: FSKD, pp. 664–668 (2013) Sunzhong, L.V., Jiang, H., Zhao, L., Wang, D., Fan, M.: Manifold based fisher method for semi-supervised feature selection. In: FSKD, pp. 664–668 (2013)
24.
go back to reference Tabakhi, S., Moradi, P., Akhlaghian, F.: An unsupervised feature selection algorithm based on ant colony optimization. Eng. Appl. Artif. Intell. 32, 112–123 (2014)CrossRef Tabakhi, S., Moradi, P., Akhlaghian, F.: An unsupervised feature selection algorithm based on ant colony optimization. Eng. Appl. Artif. Intell. 32, 112–123 (2014)CrossRef
25.
go back to reference Unler, A., Murat, A., Chinnam, R.B.: mr2PSO: a maximum relevance minimum redundancy feature selection method based on swarm intelligence for support vector machine classification. Inf. Sci. 181(20), 4625–4641 (2011)CrossRef Unler, A., Murat, A., Chinnam, R.B.: mr2PSO: a maximum relevance minimum redundancy feature selection method based on swarm intelligence for support vector machine classification. Inf. Sci. 181(20), 4625–4641 (2011)CrossRef
26.
go back to reference Wang, D., Nie, F., Huang, H.: Unsupervised feature selection via unified trace ratio formulation and K-means clustering (TRACK). In: Calders, T., Esposito, F., Hüllermeier, E., Meo, R. (eds.) ECML PKDD 2014. LNCS (LNAI), vol. 8726, pp. 306–321. Springer, Heidelberg (2014). doi:10.1007/978-3-662-44845-8_20 Wang, D., Nie, F., Huang, H.: Unsupervised feature selection via unified trace ratio formulation and K-means clustering (TRACK). In: Calders, T., Esposito, F., Hüllermeier, E., Meo, R. (eds.) ECML PKDD 2014. LNCS (LNAI), vol. 8726, pp. 306–321. Springer, Heidelberg (2014). doi:10.​1007/​978-3-662-44845-8_​20
27.
go back to reference Wang, J.Y., Yao, J., Sun, Y.: Semi-supervised local-learning-based feature selection. In: IJCNN, pp. 1942–1948 (2014) Wang, J.Y., Yao, J., Sun, Y.: Semi-supervised local-learning-based feature selection. In: IJCNN, pp. 1942–1948 (2014)
28.
go back to reference Wen, J., Lai, Z., Wong, W.K., Cui, J., Wan, M.: Optimal feature selection for robust classification via \(\ell _{2,1}\)-norms regularization. In: ICPR, pp. 517–521 (2014) Wen, J., Lai, Z., Wong, W.K., Cui, J., Wan, M.: Optimal feature selection for robust classification via \(\ell _{2,1}\)-norms regularization. In: ICPR, pp. 517–521 (2014)
29.
go back to reference Wu, X., Zhang, C., Zhang, S.: Efficient mining of both positive and negative association rules. ACM Trans. Inf. Syst. 22(3), 381–405 (2004)CrossRef Wu, X., Zhang, C., Zhang, S.: Efficient mining of both positive and negative association rules. ACM Trans. Inf. Syst. 22(3), 381–405 (2004)CrossRef
30.
go back to reference Wu, X., Zhang, S.: Synthesizing high-frequency rules from different data sources. IEEE Trans. Knowl. Data Eng. 15(2), 353–367 (2003)CrossRef Wu, X., Zhang, S.: Synthesizing high-frequency rules from different data sources. IEEE Trans. Knowl. Data Eng. 15(2), 353–367 (2003)CrossRef
31.
go back to reference Xu, Y., Song, F., Feng, G., Zhao, Y.: A novel local preserving projection scheme for use with face recognition. Expert Syst. Appl. 37(9), 6718–6721 (2010)CrossRef Xu, Y., Song, F., Feng, G., Zhao, Y.: A novel local preserving projection scheme for use with face recognition. Expert Syst. Appl. 37(9), 6718–6721 (2010)CrossRef
32.
go back to reference Yu, J., Tao, D., Wang, M.: Adaptive hypergraph learning and its application in image classification. IEEE Trans. Image Process. 21(7), 3262–3272 (2012)MathSciNetCrossRef Yu, J., Tao, D., Wang, M.: Adaptive hypergraph learning and its application in image classification. IEEE Trans. Image Process. 21(7), 3262–3272 (2012)MathSciNetCrossRef
33.
go back to reference Zhang, C., Qin, Y., Zhu, X., Zhang, J., Zhang, S.: Clustering-based missing value imputation for data preprocessing. In: IEEE International Conference on Industrial Informatics, pp. 1081–1086 (2006) Zhang, C., Qin, Y., Zhu, X., Zhang, J., Zhang, S.: Clustering-based missing value imputation for data preprocessing. In: IEEE International Conference on Industrial Informatics, pp. 1081–1086 (2006)
34.
go back to reference Zhang, S., Cheng, D., Zong, M., Gao, L.: Self-representation nearest neighbor search for classification. Neurocomputing 195, 137–142 (2016)CrossRef Zhang, S., Cheng, D., Zong, M., Gao, L.: Self-representation nearest neighbor search for classification. Neurocomputing 195, 137–142 (2016)CrossRef
35.
go back to reference Zhang, S., Li, X., Zong, M., Zhu, X., Cheng, D.: Learning k for KNN classification. ACM Transactions on Intelligent Systems and Technology (2016) Zhang, S., Li, X., Zong, M., Zhu, X., Cheng, D.: Learning k for KNN classification. ACM Transactions on Intelligent Systems and Technology (2016)
36.
go back to reference Zhang, S., Wu, X., Zhang, C.: Multi-database mining. 2, 5–13 (2003) Zhang, S., Wu, X., Zhang, C.: Multi-database mining. 2, 5–13 (2003)
37.
go back to reference Zhao, Z., Wang, L., Liu, H., Ye, J.: On similarity preserving feature selection. IEEE Trans. Knowl. Data Eng. 25(3), 619–632 (2013)CrossRef Zhao, Z., Wang, L., Liu, H., Ye, J.: On similarity preserving feature selection. IEEE Trans. Knowl. Data Eng. 25(3), 619–632 (2013)CrossRef
38.
go back to reference Zhu, P., Zuo, W., Zhang, L., Hu, Q., Shiu, S.C.: Unsupervised feature selection by regularized self-representation. Pattern Recogn. 48(2), 438–446 (2015)CrossRef Zhu, P., Zuo, W., Zhang, L., Hu, Q., Shiu, S.C.: Unsupervised feature selection by regularized self-representation. Pattern Recogn. 48(2), 438–446 (2015)CrossRef
39.
go back to reference Zhu, X., Huang, Z., Shen, H.T., Cheng, J., Xu, C.: Dimensionality reduction by mixed kernel canonical correlation analysis. Pattern Recogn. 45(8), 3003–3016 (2012)CrossRefMATH Zhu, X., Huang, Z., Shen, H.T., Cheng, J., Xu, C.: Dimensionality reduction by mixed kernel canonical correlation analysis. Pattern Recogn. 45(8), 3003–3016 (2012)CrossRefMATH
40.
go back to reference Zhu, X., Suk, H.-I., Shen, D.: Sparse discriminative feature selection for multi-class Alzheimer’s disease classification. In: Wu, G., Zhang, D., Zhou, L. (eds.) MLMI 2014. LNCS, vol. 8679, pp. 157–164. Springer, Heidelberg (2014). doi:10.1007/978-3-319-10581-9_20 Zhu, X., Suk, H.-I., Shen, D.: Sparse discriminative feature selection for multi-class Alzheimer’s disease classification. In: Wu, G., Zhang, D., Zhou, L. (eds.) MLMI 2014. LNCS, vol. 8679, pp. 157–164. Springer, Heidelberg (2014). doi:10.​1007/​978-3-319-10581-9_​20
41.
go back to reference Zhu, X., Zhang, S., Jin, Z., Zhang, Z., Xu, Z.: Missing value estimation for mixed-attribute data sets. IEEE Trans. Knowl. Data Eng. 23(1), 110–121 (2011)CrossRef Zhu, X., Zhang, S., Jin, Z., Zhang, Z., Xu, Z.: Missing value estimation for mixed-attribute data sets. IEEE Trans. Knowl. Data Eng. 23(1), 110–121 (2011)CrossRef
42.
go back to reference Zhu, X., Zhang, S., Zhang, J., Zhang, C.: Cost-sensitive imputing missing values with ordering. In: AAAI Conference on Artificial Intelligence, 22–26 July 2007, Vancouver, British Columbia, Canada, pp. 1922–1923 (2007) Zhu, X., Zhang, S., Zhang, J., Zhang, C.: Cost-sensitive imputing missing values with ordering. In: AAAI Conference on Artificial Intelligence, 22–26 July 2007, Vancouver, British Columbia, Canada, pp. 1922–1923 (2007)
43.
go back to reference Zhu, Y., Lucey, S.: Convolutional sparse coding for trajectory reconstruction. IEEE Trans. Pattern Anal. Mach. Intell. 37(3), 529–540 (2013)CrossRef Zhu, Y., Lucey, S.: Convolutional sparse coding for trajectory reconstruction. IEEE Trans. Pattern Anal. Mach. Intell. 37(3), 529–540 (2013)CrossRef
Metadata
Title
Unsupervised Hypergraph Feature Selection with Low-Rank and Self-Representation Constraints
Authors
Wei He
Xiaofeng Zhu
Yonggang Li
Rongyao Hu
Yonghua Zhu
Shichao Zhang
Copyright Year
2016
DOI
https://doi.org/10.1007/978-3-319-49586-6_12

Premium Partner