Advertisement
Published in:
2010 | OriginalPaper | Chapter
Sparse Deep Belief Net for Handwritten Digits Classification
Authors : Jiongyun Xie, Hongtao Lu, Deng Nan, Cai Nengbin
Published in: Artificial Intelligence and Computational Intelligence
Publisher: Springer Berlin Heidelberg
Activate our intelligent search to find suitable subject content or patents.
Select sections of text to find matching patents with Artificial Intelligence. powered by
Select sections of text to find additional relevant content using AI-assisted search. powered by
It has been shown that the Deep Belief Network is good at modeling input distribution, and can be trained efficiently by the greedy layer-wise unsupervised learning. Hoglak Lee et al. (2008) introduced a sparse variant of the Deep Belief Network, which applied the Gaussian linear units to model the input data with a sparsity constraint. However, it takes much more weight updates to train the RBM (Restricted Boltzmann Machine) with Gaussian visible units, and the reconstruction error is much larger than training an RBM with binary visible units. Here, we propose another version of Sparse Deep Belief Net which applies the differentiable sparse coding method to train the first level of the deep network, and then train the higher layers with RBM .This hybrid model, combining the advantage of the Deep architecture and the sparse coding model, leads to state-of-the-art performance on the classification of handwritten digits.