Yanhui Xiao
ABSTRACT
Independent Component Analysis (ICA) is an effective unsupervised tool to learn statistically independent representations. However, ICA is not only sensitive to whitening but also difficult to learn an over-complete basis set. Consequently, ICA with soft Reconstruction cost(RICA) was presented to learn sparse representations with over-complete basis even on unwhitened data. Nevertheless, this model may not be an optimal discriminative model for classification tasks, because it failed to consider the association between the training sample and its class. In this paper, we propose a supervised Discriminative ICA model with Reconstruction constraint for image classification, named DRICA. DRICA brings in class information to learn the over-complete basis by incorporating inhomogeneous representation cost constraint into the RICA framework. This constraint leads to partition the set of basis vectors into several subsets corresponding to the sample classes, where each subset could sparsely model data samples from the same class but not others. Therefore, the proposed ICA model can learn an over-complete basis and an optimal multi-class classifier jointly. Some experiments carried out on several standard image databases validate the effectiveness of DRICA for image classification.
- A. Hyvärinen, J. Karhunen, and E. Oja. Independent component analysis, volume 26. Wiley-interscience, 2001.Google Scholar
- A. Hyvärinen, J. Hurri, and P.O. Hoyer. Natural image statistics, volume 1. Springer, 2009.Google Scholar
- Q.V. Le, J. Ngiam, Z. Chen, D. Chia, P.W. Koh, and A.Y. Ng. Tiled convolutional neural networks. In NIPS, volume 23, 2010.Google Scholar
- A. Coates, H. Lee, and A.Y. Ng. An analysis of single-layer networks in unsupervised feature learning. AISTATS, 1001:48109, 2010.Google Scholar
- Y. Bengio, P. Lamblin, D. Popovici, and H. Larochelle. Greedy layer-wise training of deep networks. In NIPS, 19:153, 2007.Google ScholarDigital Library
- G.E. Hinton, S. Osindero, and Y.W. Teh. A fast learning algorithm for deep belief nets. Neural computation, 18(7):1527--1554, 2006. Google ScholarDigital Library
- Q.V. Le, A. Karpenko, J. Ngiam, and A.Y. Ng. Ica with reconstruction cost for efficient overcomplete feature learning. In NIPS, 2011.Google Scholar
- J. Yang, J. Wang, and T. Huang. Learning the sparse representation for classification. In ICME, pages 1--6. IEEE, 2011. Google ScholarDigital Library
- J. Yang, K. Yu, Y. Gong, and T. Huang. Linear spatial pyramid matching using sparse coding for image classification. In CVPR, pages 1794--1801. Ieee, 2009.Google Scholar
- Q. Zhang and B. Li. Discriminative k-svd for dictionary learning in face recognition. In CVPR. IEEE, 2010.Google ScholarCross Ref
- Z. Jiang, Z. Lin, and L.S. Davis. Learning a discriminative dictionary for sparse coding via label consistent k-svd. In CVPR, pages 1697--1704. IEEE, 2011. Google ScholarDigital Library
- M. Schmidt. minfunc. 2005.Google Scholar
- F.F. Li, R. Fergus, and P. Perona. Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories. Computer Vision and Image Understanding, 106(1):59--70, 2007. Google ScholarDigital Library
- K. Yu and T. Zhang. Improved local coordinate coding using local tangents. In ICML, 2010.Google Scholar
- A. Krizhevsky. Convolutional deep belief networks on cifar-10. Unpublished manuscript, 2010.Google Scholar
Index Terms
- Discriminative ICA model with reconstruction constraint for image classification
Recommendations
Learning a Discriminative Dictionary with CNN for Image Classification
Proceedings of the 23rd International Conference on Neural Information Processing - Volume 9948In this paper, we propose a novel framework for image recognition based on an extended sparse model. First, inspired by the impressive results of CNN over different tasks in computer vision, we use the CNN models pre-trained on large datasets to ...
Dictionary learning based on discriminative energy contribution for image classification
Learn a dictionary based on discriminative energy contribution for image classification. A linear classifier is designed in dictionary learning process for efficient classification. The ź2 norm-based dictionary learning benefits for convenient ...
A comparative study of PCA, ICA and class-conditional ICA for Naïve Bayes classifier
IWANN'07: Proceedings of the 9th international work conference on Artificial neural networksThe performance of the Naïve Bayes classifier can be improved by appropriate preprocessing procedures. This paper presents a comparative study of three preprocessing procedures, namely Principle Component Analysis (PCA), Independent Component Analysis (...
Comments