Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Published in:
Classifying Human Activities with Temporal Extension of Random Forest Read chapter Read first chapter

2016 | OriginalPaper | Chapter

A Structural Learning Method of Restricted Boltzmann Machine by Neuron Generation and Annihilation Algorithm

Authors : Shin Kamada, Takumi Ichimura

Published in: Neural Information Processing

Publisher: Springer International Publishing

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Restricted Boltzmann Machine (RBM) is a generative stochastic energy-based model of artificial neural network for unsupervised learning. The adaptive learning method that can discover the optimal number of hidden neurons according to the input space is important method in terms of the stability of energy as well as the computational cost although a traditional RBM model cannot change its network structure during learning phase. Moreover, we should consider the regularities in the sparse of network to extract explicit knowledge from the network because the trained network is often a black box. In this paper, we propose the combination method of adaptive and structural learning method of RBM with Forgetting that can discover the regularities in the trained network. We evaluated our proposed model on MNIST and CIFAR-10 datasets.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
previous chapter Fissionable Deep Neural Network
next chapter Semi-supervised Learning for Convolutional Neural Networks Using Mild Supervisory Signals
Literature
1.
2.
Hinton, G.E.: A practical guide to training restricted boltzmann machines. In: Montavon, G., Orr, G.B., Müller, K.-R. (eds.) Neural Networks: Tricks of the Trade, 2nd edn. LNCS, vol. 7700, pp. 599–619. Springer, Heidelberg (2012)CrossRef
3.
Kamada, S., Ichimura, T.: A learning method of adaptive deep belief network by using neuron generation and annihilation algorithm. In: Proceedings of 17th Annual Meeting of Self-Organizing Maps in Japanese, pp. 12.1–12.6 (2016)
4.
Ranzato, M., Boureau, Y., LeCun, Y.: Sparse feature learning for deep belief networks. In: Advances in Neural Information Processing Systems 20 (NIPS 2007), pp. 1185–1192 (2007)
5.
Ishikawa, M.: Structural learning with forgetting. Neural Netw. 9(3), 509–521 (1996)CrossRef
6.
Kamada, S., Fujii, Y., Ichimura, T.: Structural learning method of restricted Boltzmann machine with forgetting. In: Proceedings of 17th Annual Meeting of Self-Organizing Maps in Japanese, pp. 13.1–13.6 (2016)
7.
Hinton, G.E.: Training products of experts by minimizing contrastive divergence. Neural Comput. 14, 1771–1800 (2002)CrossRefMATH
8.
Carlson, D., Cevher, V., Carin, L.: Stochastic spectral descent for restricted Boltzmann machines. In: Proceedings of the Eighteenth International Conference on Artificial Intelligence and Statistics, pp. 111–119 (2015)
9.
Kamada, S., Ichimura, T., Fujii, Y.: A consideration of convergence of energy function in restricted Boltzmann machine by Lipschitz continuity. In: Proceedings of IEEE SMC Hiroshima Chapter YRW 2015 in Japanese, pp. 53–56 (2015)
10.
Ichimura, T., Yoshida, K. (eds.): Knowledge-Based Intelligent Systems for Health Care. Advanced Knowledge International (ISBN 0-9751004-4-0) (2004)
11.
12.
Krizhevsky, A.: Learning multiple layers of features from tiny images. Master of thesis, University of Toronto (2009)
13.
Dieleman, S., Schrauwen, B.: Accelerating sparse restricted Boltzmann machine training using non-Gaussianity measures. In: Deep Learning and Unsupervised Feature Learning (NIPS-2012) (2012)
14.
15.
Kamada, S., Ichimura, T.: An adaptive learning method with forgetting in deep belief network. In: Proceedings of 9th SICE Symposium on Computational Intelligence, pp. 92–96 (2016)
16.
Kamada, S., Ichimura, T.: An adaptive learning method of restricted Boltzmann machine by neuron generation and annihilation algorithm. In: Proceedings of 2016 IEEE SMC (SMC 2016) (accepted)
Metadata
Title
A Structural Learning Method of Restricted Boltzmann Machine by Neuron Generation and Annihilation Algorithm
Authors
Shin Kamada
Takumi Ichimura
Copyright Year
2016
Book
Neural Information Processing

Print ISBN: 978-3-319-46680-4

Electronic ISBN: 978-3-319-46681-1

Copyright Year: 2016

DOI
https://doi.org/10.1007/978-3-319-46681-1_45

Premium Partner

    Image Credits