Skip to main content
Top
Published in: Neural Computing and Applications 10/2019

13-04-2018 | Original Article

Exploiting label consistency in structured sparse representation for classification

Authors: Yan Huang, Yuhui Quan, Tao Liu, Yong Xu

Published in: Neural Computing and Applications | Issue 10/2019

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Sparse representation with adaptive dictionaries has emerged as a promising tool in computer vision and pattern analysis. While standard sparsity promoted by \(\ell _0\) or \(\ell _1\) regularization has been widely used, recent approaches seek for kinds of structured sparsity to improve the discriminability of sparse codes. For classification, label consistency is one useful concept regarding structured sparsity, which relates class labels to dictionary atoms for generating discriminative sparsity patterns. Motivated by the limitations of existing label-consistent regularization methods, in this paper, we investigate the exploitation of label consistency and propose an effective sparse coding approach. The proposed approach enforces the sparse approximation of a label consistency matrix by sparse code during dictionary learning, which encourages the supports of sparse codes to be consistent for intra-class signals and distinct for inter-class signals. Thus, the learned dictionary can induce discriminative sparsity patterns when used in sparse coding. Moreover, the proposed method is computationally efficient, as the label consistency regularization developed in our method brings very little additional computational cost in solving the related sparse coding problem. The effectiveness of the proposed method is demonstrated with several recognition tasks, and the experimental results show that our method is very competitive with some state-of-the-art approaches.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Footnotes
1
Such a kind of sparsity is often referred to as structured sparsity.
 
2
This is often true if the corresponding signal \({\varvec{y}}_1\) and \({\varvec{y}}_2\) are from different subclasses.
 
3
For rigorous proof, the sequence \(\{{\varvec{C}}^{(k)}\}_k\) is required to be bounded, which can be guaranteed by simple projection of \({\varvec{C}}^{(k)}\) at each iteration.
 
Literature
1.
go back to reference Aharon M, Elad M, Bruckstein A (2006) K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation. Trans Signal Process 54(11):4311–4322CrossRef Aharon M, Elad M, Bruckstein A (2006) K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation. Trans Signal Process 54(11):4311–4322CrossRef
2.
go back to reference Bagheri MA, Gao Q, Escalera S, Moeslund TB, Ren H, Etemad E (2017) Locality regularized group sparse coding for action recognition. Comput Vis Image Underst 158:106–114CrossRef Bagheri MA, Gao Q, Escalera S, Moeslund TB, Ren H, Etemad E (2017) Locality regularized group sparse coding for action recognition. Comput Vis Image Underst 158:106–114CrossRef
3.
go back to reference Bao C, Ji H, Quan Y, Shen Z (2016) Dictionary learning for sparse coding: algorithms and convergence analysis. Trans Pattern Anal Mach Intell 38(7):1356–1369CrossRef Bao C, Ji H, Quan Y, Shen Z (2016) Dictionary learning for sparse coding: algorithms and convergence analysis. Trans Pattern Anal Mach Intell 38(7):1356–1369CrossRef
4.
go back to reference Boureau YL, Bach F, LeCun Y, Ponce J (2010) Learning mid-level features for recognition. In: Conference on computer vision and pattern recognition, IEEE, pp 2559–2566 Boureau YL, Bach F, LeCun Y, Ponce J (2010) Learning mid-level features for recognition. In: Conference on computer vision and pattern recognition, IEEE, pp 2559–2566
5.
go back to reference Cai S, Zuo W, Zhang L, Feng X, Wang P (2014) Support vector guided dictionary learning. In: ECCV, Springer, pp 624–639 Cai S, Zuo W, Zhang L, Feng X, Wang P (2014) Support vector guided dictionary learning. In: ECCV, Springer, pp 624–639
6.
go back to reference Chi YT, Ali M, Rajwade A, Ho J (2013) Block and group regularized sparse modeling for dictionary learning. In: Conference on computer vision and pattern recognition, IEEE, pp 377–382 Chi YT, Ali M, Rajwade A, Ho J (2013) Block and group regularized sparse modeling for dictionary learning. In: Conference on computer vision and pattern recognition, IEEE, pp 377–382
7.
go back to reference Gao Q, Wang Q, Huang Y, Gao X, Hong X, Zhang H (2015) Dimensionality reduction by integrating sparse representation and fisher criterion and its applications. Trans Image Process 24(12):5684–5695MathSciNetCrossRef Gao Q, Wang Q, Huang Y, Gao X, Hong X, Zhang H (2015) Dimensionality reduction by integrating sparse representation and fisher criterion and its applications. Trans Image Process 24(12):5684–5695MathSciNetCrossRef
8.
go back to reference Gao S, Tsang IW, Chia LT, Zhao P (2010) Local features are not lonely–Laplacian sparse coding for image classification. In: Conference on computer vision and pattern recognition, IEEE, pp 3555–3561 Gao S, Tsang IW, Chia LT, Zhao P (2010) Local features are not lonely–Laplacian sparse coding for image classification. In: Conference on computer vision and pattern recognition, IEEE, pp 3555–3561
9.
go back to reference Gao Y, Ma J, Yuille AL (2017) Semi-supervised sparse representation based classification for face recognition with insufficient labeled samples. Trans Image Process 26(5):2545–2560MathSciNetCrossRef Gao Y, Ma J, Yuille AL (2017) Semi-supervised sparse representation based classification for face recognition with insufficient labeled samples. Trans Image Process 26(5):2545–2560MathSciNetCrossRef
10.
go back to reference Ghanem B, Ahuja N (2010) Maximum margin distance learning for dynamic texture recognition. In: European conference on computer vision, Springer, pp 223–236 Ghanem B, Ahuja N (2010) Maximum margin distance learning for dynamic texture recognition. In: European conference on computer vision, Springer, pp 223–236
11.
go back to reference Ghanem B, Ahuja N (2010) Maximum margin distance learning for dynamic texture recognition. In: European conference on computer vision, Springer, pp 223–236 Ghanem B, Ahuja N (2010) Maximum margin distance learning for dynamic texture recognition. In: European conference on computer vision, Springer, pp 223–236
12.
go back to reference Huang K, Aviyente S (2006) Sparse representation for signal classification. In: Advances in neural information processing systems, pp 609–616 Huang K, Aviyente S (2006) Sparse representation for signal classification. In: Advances in neural information processing systems, pp 609–616
13.
go back to reference Jacob L, Obozinski G, Vert JP (2009) Group lasso with overlap and graph lasso. In: International conference on machine learning, ACM, pp 433–440 Jacob L, Obozinski G, Vert JP (2009) Group lasso with overlap and graph lasso. In: International conference on machine learning, ACM, pp 433–440
14.
go back to reference Jenatton R, Audibert JY, Bach F (2011) Structured variable selection with sparsity-inducing norms. J Mach Learn Res 12:2777–2824MathSciNetMATH Jenatton R, Audibert JY, Bach F (2011) Structured variable selection with sparsity-inducing norms. J Mach Learn Res 12:2777–2824MathSciNetMATH
15.
go back to reference Jiang Z, Lin Z, Davis LS (2011) Learning a discriminative dictionary for sparse coding via label consistent K-SVD. In: Conference on computer vision and pattern recognition, IEEE, pp 1697–1704 Jiang Z, Lin Z, Davis LS (2011) Learning a discriminative dictionary for sparse coding via label consistent K-SVD. In: Conference on computer vision and pattern recognition, IEEE, pp 1697–1704
16.
go back to reference Jiang Z, Zhang G, Davis LS (2012) Submodular dictionary learning for sparse coding. In: Proceedings of IEEE conference on computer vision and pattern recognition, IEEE, pp 3418–3425 Jiang Z, Zhang G, Davis LS (2012) Submodular dictionary learning for sparse coding. In: Proceedings of IEEE conference on computer vision and pattern recognition, IEEE, pp 3418–3425
17.
go back to reference Jiang Z, Lin Z, Davis L (2013) Label consistent K-SVD: learning a discriminative dictionary for recognition. Trans Pattern Anal Mach Intell 35(11):2651–2664CrossRef Jiang Z, Lin Z, Davis L (2013) Label consistent K-SVD: learning a discriminative dictionary for recognition. Trans Pattern Anal Mach Intell 35(11):2651–2664CrossRef
18.
go back to reference Kim S, Xing EP (2010) Tree-guided group lasso for multi-task regression with structured sparsity. In: International conference on machine learning, pp 543–550 Kim S, Xing EP (2010) Tree-guided group lasso for multi-task regression with structured sparsity. In: International conference on machine learning, pp 543–550
19.
go back to reference Kong S, Wang D (2012) A dictionary learning approach for classification: separating the particularity and the commonality. In: ECCV, Springer, pp 186–199 Kong S, Wang D (2012) A dictionary learning approach for classification: separating the particularity and the commonality. In: ECCV, Springer, pp 186–199
20.
go back to reference Kviatkovsky I, Gabel M, Rivlin E, Shimshoni I (2017) On the equivalence of the LC-KSVD and the D-KSVD algorithms. Trans Pattern Anal Mach Intell 39(2):411–416CrossRef Kviatkovsky I, Gabel M, Rivlin E, Shimshoni I (2017) On the equivalence of the LC-KSVD and the D-KSVD algorithms. Trans Pattern Anal Mach Intell 39(2):411–416CrossRef
21.
go back to reference Lazebnik S, Schmid C, Ponce J (2006) Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In: Conference on computer vision and pattern recognition, vol 2. IEEE, pp 2169–2178 Lazebnik S, Schmid C, Ponce J (2006) Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In: Conference on computer vision and pattern recognition, vol 2. IEEE, pp 2169–2178
22.
go back to reference Lian XC, Li Z, Lu BL, Zhang L (2010) Max-margin dictionary learning for multiclass image categorization. In: European conference on computer vision, Springer, pp 157–170 Lian XC, Li Z, Lu BL, Zhang L (2010) Max-margin dictionary learning for multiclass image categorization. In: European conference on computer vision, Springer, pp 157–170
23.
go back to reference Lian XC, Li Z, Wang C, Lu BL, Zhang L (2010) Probabilistic models for supervised dictionary learning. In: Conference on computer vision and pattern recognition, IEEE, pp 2305–2312 Lian XC, Li Z, Wang C, Lu BL, Zhang L (2010) Probabilistic models for supervised dictionary learning. In: Conference on computer vision and pattern recognition, IEEE, pp 2305–2312
24.
go back to reference Liu J, Ye J (2010) Moreau–Yosida regularization for grouped tree structure learning. Adv Neural Inf Process Syst 23:1459–1467 Liu J, Ye J (2010) Moreau–Yosida regularization for grouped tree structure learning. Adv Neural Inf Process Syst 23:1459–1467
25.
go back to reference Lu J, Wang G, Deng W, Moulin P (2014) Simultaneous feature and dictionary learning for image set based face recognition. In: European conference on computer vision, Springer, pp 265–280 Lu J, Wang G, Deng W, Moulin P (2014) Simultaneous feature and dictionary learning for image set based face recognition. In: European conference on computer vision, Springer, pp 265–280
26.
go back to reference Mairal J, Bach F, Ponce J, Sapiro G, Zisserman A (2008) Discriminative learned dictionaries for local image analysis. In: Conference on computer vision and pattern recognition, IEEE, pp 1–8 Mairal J, Bach F, Ponce J, Sapiro G, Zisserman A (2008) Discriminative learned dictionaries for local image analysis. In: Conference on computer vision and pattern recognition, IEEE, pp 1–8
27.
go back to reference Mairal J, Ponce J, Sapiro G, Zisserman A, Bach FR (2009) Supervised dictionary learning. In: Advances in neural information processing systems, pp 1033–1040 Mairal J, Ponce J, Sapiro G, Zisserman A, Bach FR (2009) Supervised dictionary learning. In: Advances in neural information processing systems, pp 1033–1040
28.
go back to reference Majumdar A (2015) Discriminative label consistent dictionary learning. In: 2015 IEEE international conference on image processing (ICIP), IEEE, pp 1016–1020 Majumdar A (2015) Discriminative label consistent dictionary learning. In: 2015 IEEE international conference on image processing (ICIP), IEEE, pp 1016–1020
29.
go back to reference Marsousi M, Li X, Plataniotis KN (2016) Shape-included label-consistent discriminative dictionary learning: an approach to detect and segment multi-class objects in images. In: International conference on image processing, IEEE, pp 729–733 Marsousi M, Li X, Plataniotis KN (2016) Shape-included label-consistent discriminative dictionary learning: an approach to detect and segment multi-class objects in images. In: International conference on image processing, IEEE, pp 729–733
30.
go back to reference Martinez AM (1998) The AR face database. CVC Technical report 24 Martinez AM (1998) The AR face database. CVC Technical report 24
31.
go back to reference Péteri R, Fazekas S, Huiskes MJ (2010) DynTex : a comprehensive database of dynamic textures. Pattern Recogn Lett 31:1627–1632CrossRef Péteri R, Fazekas S, Huiskes MJ (2010) DynTex : a comprehensive database of dynamic textures. Pattern Recogn Lett 31:1627–1632CrossRef
32.
go back to reference Pham DS, Venkatesh S (2008) Joint learning and dictionary construction for pattern recognition. In: Conference on computer vision and pattern recognition, IEEE, pp 1–8 Pham DS, Venkatesh S (2008) Joint learning and dictionary construction for pattern recognition. In: Conference on computer vision and pattern recognition, IEEE, pp 1–8
33.
go back to reference Quan Y, Ji H, Shen Z (2014) Data-driven multi-scale non-local wavelet frame construction and image recovery. J Sci Comput 63:307–329MathSciNetCrossRef Quan Y, Ji H, Shen Z (2014) Data-driven multi-scale non-local wavelet frame construction and image recovery. J Sci Comput 63:307–329MathSciNetCrossRef
34.
go back to reference Quan Y, Xu Y, Sun Y, Huang Y (2016) Supervised dictionary learning with multiple classifier integration. Pattern Recogn 55:247–260CrossRef Quan Y, Xu Y, Sun Y, Huang Y (2016) Supervised dictionary learning with multiple classifier integration. Pattern Recogn 55:247–260CrossRef
35.
go back to reference Quan Y, Xu Y, Sun Y, Huang Y, Ji H (2016) Sparse coding for classification via discrimination ensemble. In: Conference on computer vision and pattern recognition, pp 5839–5847 Quan Y, Xu Y, Sun Y, Huang Y, Ji H (2016) Sparse coding for classification via discrimination ensemble. In: Conference on computer vision and pattern recognition, pp 5839–5847
36.
go back to reference Ramirez I, Sprechmann P, Sapiro G (2010) Classification and clustering via dictionary learning with structured incoherence and shared features. In: Conference on computer vision and pattern recognition, IEEE, pp 3501–3508 Ramirez I, Sprechmann P, Sapiro G (2010) Classification and clustering via dictionary learning with structured incoherence and shared features. In: Conference on computer vision and pattern recognition, IEEE, pp 3501–3508
37.
go back to reference Rosenblum K, Zelnik-Manor L, Eldar Y (2010) Dictionary optimization for block-sparse representations. In: Association for the advancement of artificial intelligence fall symposium: manifold learning and its applications, pp 50–58 Rosenblum K, Zelnik-Manor L, Eldar Y (2010) Dictionary optimization for block-sparse representations. In: Association for the advancement of artificial intelligence fall symposium: manifold learning and its applications, pp 50–58
38.
go back to reference Sprechmann P, Ramirez I, Sapiro G, Eldar YC (2011) C-HiLasso: a collaborative hierarchical sparse modeling framework. Trans Signal Process 59(9):4183–4198MathSciNetCrossRef Sprechmann P, Ramirez I, Sapiro G, Eldar YC (2011) C-HiLasso: a collaborative hierarchical sparse modeling framework. Trans Signal Process 59(9):4183–4198MathSciNetCrossRef
39.
go back to reference Sun Y, Liu Q, Tang J, Tao D (2014) Learning discriminative dictionary for group sparse representation. Trans Image Process 23(9):3816–3828MathSciNetCrossRef Sun Y, Liu Q, Tang J, Tao D (2014) Learning discriminative dictionary for group sparse representation. Trans Image Process 23(9):3816–3828MathSciNetCrossRef
40.
go back to reference Wang J, Yang J, Yu K, Lv F, Huang T, Gong Y (2010) Locality-constrained linear coding for image classification. In: Conference on computer vision and pattern recognition, IEEE, pp 3360–3367 Wang J, Yang J, Yu K, Lv F, Huang T, Gong Y (2010) Locality-constrained linear coding for image classification. In: Conference on computer vision and pattern recognition, IEEE, pp 3360–3367
41.
go back to reference Wang X, Yang M, Shen L (2016) Structured regularized robust coding for face recognition. Neurocomputing 216:18–27CrossRef Wang X, Yang M, Shen L (2016) Structured regularized robust coding for face recognition. Neurocomputing 216:18–27CrossRef
42.
go back to reference Wright J, Yang AY, Ganesh A, Sastry SS, Ma Y (2009) Robust face recognition via sparse representation. Trans Pattern Anal Mach Intell 31(2):210–227CrossRef Wright J, Yang AY, Ganesh A, Sastry SS, Ma Y (2009) Robust face recognition via sparse representation. Trans Pattern Anal Mach Intell 31(2):210–227CrossRef
43.
go back to reference Xu Y, Quan Y, Ling H, Ji H (2011) Dynamic texture classification using dynamic fractal analysis. In: International conference on computer vision, IEEE, pp 1219–1226 Xu Y, Quan Y, Ling H, Ji H (2011) Dynamic texture classification using dynamic fractal analysis. In: International conference on computer vision, IEEE, pp 1219–1226
44.
go back to reference Yang J, Yu K, Gong Y, Huang T (2009) Linear spatial pyramid matching using sparse coding for image classification. In: Conference on computer vision and pattern recognition, IEEE, pp 1794–1801 Yang J, Yu K, Gong Y, Huang T (2009) Linear spatial pyramid matching using sparse coding for image classification. In: Conference on computer vision and pattern recognition, IEEE, pp 1794–1801
45.
go back to reference Yang J, Yu K, Huang T (2010) Supervised translation-invariant sparse coding. In: Conference on computer vision and pattern recognition, IEEE, pp 3517–3524 Yang J, Yu K, Huang T (2010) Supervised translation-invariant sparse coding. In: Conference on computer vision and pattern recognition, IEEE, pp 3517–3524
46.
go back to reference Yang J, Zhang L, Xu Y, Jy Yang (2012) Beyond sparsity: the role of L1-optimizer in pattern classification. Pattern Recogn 45(3):1104–1118CrossRef Yang J, Zhang L, Xu Y, Jy Yang (2012) Beyond sparsity: the role of L1-optimizer in pattern classification. Pattern Recogn 45(3):1104–1118CrossRef
47.
go back to reference Yang M, Zhang D, Feng X (2011) Fisher discrimination dictionary learning for sparse representation. In: International conference on computer vision, IEEE, pp 543–550 Yang M, Zhang D, Feng X (2011) Fisher discrimination dictionary learning for sparse representation. In: International conference on computer vision, IEEE, pp 543–550
48.
go back to reference Yang M, Dai D, Shen L, Van Gool L (2014) Latent dictionary learning for sparse representation based classification. In: Conference on computer vision and pattern recognition, pp 4138–4145 Yang M, Dai D, Shen L, Van Gool L (2014) Latent dictionary learning for sparse representation based classification. In: Conference on computer vision and pattern recognition, pp 4138–4145
49.
go back to reference Yang M, Zhang L, Feng X, Zhang D (2014) Sparse representation based fisher discrimination dictionary learning for image classification. Int J Comput Vision 109(3):209–232MathSciNetCrossRef Yang M, Zhang L, Feng X, Zhang D (2014) Sparse representation based fisher discrimination dictionary learning for image classification. Int J Comput Vision 109(3):209–232MathSciNetCrossRef
50.
go back to reference Zhang D, Liu P, Zhang K, Zhang H, Wang Q, Jing X (2015) Class relatedness oriented-discriminative dictionary learning for multiclass image classification. Pattern Recognit 59:168–175CrossRef Zhang D, Liu P, Zhang K, Zhang H, Wang Q, Jing X (2015) Class relatedness oriented-discriminative dictionary learning for multiclass image classification. Pattern Recognit 59:168–175CrossRef
51.
go back to reference Zhang H, Wu QJ, Chow TW, Zhao M (2012) A two-dimensional neighborhood preserving projection for appearance-based face recognition. Pattern Recogn 45(5):1866–1876CrossRef Zhang H, Wu QJ, Chow TW, Zhao M (2012) A two-dimensional neighborhood preserving projection for appearance-based face recognition. Pattern Recogn 45(5):1866–1876CrossRef
52.
go back to reference Zhang Q, Li B (2010) Discriminative K-SVD for dictionary learning in face recognition. In: Conference on computer vision and pattern recognition, IEEE, pp 2691–2698 Zhang Q, Li B (2010) Discriminative K-SVD for dictionary learning in face recognition. In: Conference on computer vision and pattern recognition, IEEE, pp 2691–2698
53.
go back to reference Zhao G, Pietikainen M (2007) Dynamic texture recognition using local binary patterns with an application to facial expressions. Trans Pattern Anal Mach Intell 29(6):915–928CrossRef Zhao G, Pietikainen M (2007) Dynamic texture recognition using local binary patterns with an application to facial expressions. Trans Pattern Anal Mach Intell 29(6):915–928CrossRef
54.
go back to reference Zhou N, Shen Y, Peng J, Fan J (2012) Learning inter-related visual dictionary for object recognition. In: Conference on computer vision and pattern recognition, IEEE, pp 3490–3497 Zhou N, Shen Y, Peng J, Fan J (2012) Learning inter-related visual dictionary for object recognition. In: Conference on computer vision and pattern recognition, IEEE, pp 3490–3497
Metadata
Title
Exploiting label consistency in structured sparse representation for classification
Authors
Yan Huang
Yuhui Quan
Tao Liu
Yong Xu
Publication date
13-04-2018
Publisher
Springer London
Published in
Neural Computing and Applications / Issue 10/2019
Print ISSN: 0941-0643
Electronic ISSN: 1433-3058
DOI
https://doi.org/10.1007/s00521-018-3479-0

Other articles of this Issue 10/2019

Neural Computing and Applications 10/2019 Go to the issue

Premium Partner