Skip to main content

2014 | OriginalPaper | Buchkapitel

A Hierarchical Approach for Handwritten Digit Recognition Using Sparse Autoencoder

verfasst von : An T. Duong, Hai T. Phan, Nam Do-Hoang Le, Son T. Tran

Erschienen in: Issues and Challenges of Intelligent Systems and Computational Intelligence

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Higher level features learning algorithms have been applied on handwritten digit recognition and got more promising results than just using raw intensity values with classification algorithms. However, the approaches of these algorithms still not take the advantage of specific characteristics of data. We propose a new method to learn higher level features from specific characteristics of data using sparse autoencoder. The main key of our appoarch is to divide the handwritten digits into subsets corresponding to specific characteristics. The experimental results show that the proposed method achieves lower error rates and time complexity than the original approach of sparse autoencoder. The results also show that the more correlated characteristics we define, the better higher level features we learn.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Bagnell, J.A., Bradley, D.M.: Differentiable sparse coding. In: NIPS. pp. 113–120 (2008) Bagnell, J.A., Bradley, D.M.: Differentiable sparse coding. In: NIPS. pp. 113–120 (2008)
2.
Zurück zum Zitat Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: CVPR. vol. 1, pp. 886–893 (2005) Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: CVPR. vol. 1, pp. 886–893 (2005)
3.
Zurück zum Zitat DeCoste, D., Schölkopf, B.: Training invariant support vector machines. Mach. Learn. 46(1–3), 161–190 (2002) DeCoste, D., Schölkopf, B.: Training invariant support vector machines. Mach. Learn. 46(1–3), 161–190 (2002)
4.
Zurück zum Zitat Grosse, R., Raina, R., Kwong, H., Andrew, Y.Ng.: Shift-invariance sparse coding for audio classification. CoRR, abs/1206.5241, 2012. Grosse, R., Raina, R., Kwong, H., Andrew, Y.Ng.: Shift-invariance sparse coding for audio classification. CoRR, abs/1206.5241, 2012.
5.
6.
Zurück zum Zitat Hinton, G.E., Osindero, S., Teh, Y.W.: A fast learning algorithm for deep belief nets. Neural Comput. 18(7), 1527–1554 (2006)CrossRefMATHMathSciNet Hinton, G.E., Osindero, S., Teh, Y.W.: A fast learning algorithm for deep belief nets. Neural Comput. 18(7), 1527–1554 (2006)CrossRefMATHMathSciNet
7.
Zurück zum Zitat Lennig M., Hunt, M., Mermelstein, P.: Experiments in syllable-based recognition of continuous speech. In: Proceedings of International Conference on Acoustics, Speech and, Signal Processing, pp. 880–883 (1996) Lennig M., Hunt, M., Mermelstein, P.: Experiments in syllable-based recognition of continuous speech. In: Proceedings of International Conference on Acoustics, Speech and, Signal Processing, pp. 880–883 (1996)
8.
Zurück zum Zitat Huth, A.G., Gallant, J.L., Vu, A.T., Nishimoto, S.: A continuous semantic space describes the representation of thousands of object and action categories across the human brain. Neuron 76(6), 1210–1224 (2012)CrossRef Huth, A.G., Gallant, J.L., Vu, A.T., Nishimoto, S.: A continuous semantic space describes the representation of thousands of object and action categories across the human brain. Neuron 76(6), 1210–1224 (2012)CrossRef
9.
Zurück zum Zitat LeCun, Y., Bottou, L., Bengio, Y., Haffner, H.: Gradient-based learning applied to document recognition. In: Proceedings of the IEEE. vol. 86, pp. 2278–2324 (1998) LeCun, Y., Bottou, L., Bengio, Y., Haffner, H.: Gradient-based learning applied to document recognition. In: Proceedings of the IEEE. vol. 86, pp. 2278–2324 (1998)
10.
Zurück zum Zitat Lee, H., Battle, A., Raina, R., Andrew, Y.Ng.: Efficient sparse coding algorithms. In: Advances in Neural Information Processing Systems. vol. 19, pp. 801–808 (2007) Lee, H., Battle, A., Raina, R., Andrew, Y.Ng.: Efficient sparse coding algorithms. In: Advances in Neural Information Processing Systems. vol. 19, pp. 801–808 (2007)
11.
Zurück zum Zitat Lee, H., Grosse, R., Ranganath, R., Andrew, Y.Ng.: Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In: ICML. p. 77 (2009) Lee, H., Grosse, R., Ranganath, R., Andrew, Y.Ng.: Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In: ICML. p. 77 (2009)
12.
Zurück zum Zitat Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60(2), 91–110 (2004) Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60(2), 91–110 (2004)
13.
Zurück zum Zitat Martens, J.: Deep learning via hessian-free optimization. In: Fürnkranz, J., Joachims, T.,(eds.) Proceedings of the 27th International Conference on Machine Learning (ICML-10), Haifa, Israel, pp. 735–742 June 2010 Martens, J.: Deep learning via hessian-free optimization. In: Fürnkranz, J., Joachims, T.,(eds.) Proceedings of the 27th International Conference on Machine Learning (ICML-10), Haifa, Israel, pp. 735–742 June 2010
14.
Zurück zum Zitat Olshausen, B.A., Field, D.J.: Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607–609 (1996)CrossRef Olshausen, B.A., Field, D.J.: Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607–609 (1996)CrossRef
15.
Zurück zum Zitat Raina, R., Battle, A., Lee, H., Packer, B., Andrew Y.Ng.: Self-taught learning: Transfer learning from unlabeled data. In: ICML ’07: Proceedings of the 24th International Conference on, Machine learning (2007) Raina, R., Battle, A., Lee, H., Packer, B., Andrew Y.Ng.: Self-taught learning: Transfer learning from unlabeled data. In: ICML ’07: Proceedings of the 24th International Conference on, Machine learning (2007)
16.
Zurück zum Zitat Simard, P., Steinkraus, D., Platt, J.C.: Best practices for convolutional neural networks applied to visual document analysis. In: ICDAR, pp. 958–962 (2003) Simard, P., Steinkraus, D., Platt, J.C.: Best practices for convolutional neural networks applied to visual document analysis. In: ICDAR, pp. 958–962 (2003)
17.
Zurück zum Zitat Socher, R., Eric, H., Pennin, J., Andrew, Y.Ng, Manning, C.D.: Dynamic pooling and unfolding recursive autoencoders for paraphrase detection. In: Shawe-Taylor, J., Zemel, R.S., Bartlett, P., Pereira, F.C.N., Weinberger, K.Q., (eds.) Proceeding of 24th Advances in Neural Information Processing Systems, pp. 801–809 (2011) Socher, R., Eric, H., Pennin, J., Andrew, Y.Ng, Manning, C.D.: Dynamic pooling and unfolding recursive autoencoders for paraphrase detection. In: Shawe-Taylor, J., Zemel, R.S., Bartlett, P., Pereira, F.C.N., Weinberger, K.Q., (eds.) Proceeding of 24th Advances in Neural Information Processing Systems, pp. 801–809 (2011)
18.
Zurück zum Zitat Kai. Y., Zhang, T., Gong, Y.: Nonlinear learning using local coordinate coding. In: Bengio, Y., Schuurmans, D., Lafferty, J.D., Williams, C.K.I., Culotta, A. (eds.) Proceeding of 24th Advances in Neural Information Processing Systems, NIPS, pp. 2223–2231 (2009) (Curran Associates Inc.) Kai. Y., Zhang, T., Gong, Y.: Nonlinear learning using local coordinate coding. In: Bengio, Y., Schuurmans, D., Lafferty, J.D., Williams, C.K.I., Culotta, A. (eds.) Proceeding of 24th Advances in Neural Information Processing Systems, NIPS, pp. 2223–2231 (2009) (Curran Associates Inc.)
Metadaten
Titel
A Hierarchical Approach for Handwritten Digit Recognition Using Sparse Autoencoder
verfasst von
An T. Duong
Hai T. Phan
Nam Do-Hoang Le
Son T. Tran
Copyright-Jahr
2014
DOI
https://doi.org/10.1007/978-3-319-03206-1_10