Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
International Journal of Machine Learning and Cybernetics
Published in: International Journal of Machine Learning and Cybernetics 7/2019

20-06-2018 | Original Article

Deep Boltzmann machine for nonlinear system modelling

Authors: Wen Yu, Erick de la Rosa

Published in: International Journal of Machine Learning and Cybernetics | Issue 7/2019

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Deep Boltzmann machine (DBM) has been successfully applied in classification, regression and time series modeling. For nonlinear system modelling, DBM should also have many advantages over the other neural networks, such as input features extraction and noise tolerance. In this paper, we use DBM to model nonlinear systems by calculating the probability distributions of the input and output. Two novel weight updating algorithms are proposed to obtain these distributions. We use binary encoding and conditional probability transformation methods. The proposed methods are validated with two benchmark nonlinear systems.
previous article Predicting algal appearance on mortar surface with ensembles of adaptive neuro fuzzy models: a comparative study of ensemble strategies
next article Optimal pricing decision for supply chains with risk sensitivity and human estimation

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
Show more products
Literature
1.
LeCun Y, Bengio Y, Hinton GE (2015) Deep learning. Nature 521(7553):436–444CrossRef
2.
Wang Z, Wang X (2018) A deep stochastic weight assignment network and its application to chess playing. J Parall Distrib Comput 117:205–211CrossRef
3.
4.
LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324CrossRef
5.
Salakhutdinov R, Hinton GE (2009) Deep boltzmann machines. In: 12th International conference on artificial intelligence and statistics (AISTATS). Clearwater Beach, Florida, USA
6.
7.
Larochelle H, Bengio Y (2008) Classification using discriminative restricted Boltzmann machines. In: 25th International conference on machine learning, Helsinki, Finland, pp 536–543
8.
Le Roux N, Bengio Y (2008) Representational power of restricted Boltzmann machines and deep belief networks. Neural Comput 20:1631–1649MathSciNetCrossRefMATH
9.
Erhan D, Bengio Y, Courville A, Manzagol P-A, Vincent P (2010) Why does unsupervised pre-training help deep learning? J Mach Learn Res 11:625–660MathSciNetMATH
10.
11.
Hinton GE, Sejnowski TJ (1986) Learning and relearning in Boltzmann machines. In: Parallel distributed processing: explorations in the microstructure of cognition, vol 1: foundations, pp 282–317, MIT Press, Cambridge, MA
12.
Qiu L, Zhang L, Ren Y, Suganthan PN, Amaratunga G (2014) Ensemble deep learning for regression and time series forecasting. In: 2014 IEEE symposium on computational intelligence in ensemble learning (CIEL), pp 1–6, Orlando, FL, USA
13.
Busseti E, Osband I, Wong S (2012) Deep learning for time series modeling. Technical report, Stanford University
14.
Längkvist M, Karlsson L, Loutfi A (2014) A review of unsupervised feature learning and deep learning for time-series modeling. Pattern Recogn Lett 42:11–24CrossRef
15.
Romeu P et al (2013) Time-series forecasting time-series, of indoor temperature using pre-trained deep neural networks. Artificial Neural Networks and Machine Learning-ICANN. Springer, Berlin Heidelberg, pp 451–458
16.
Ljung L (1987) System identification-theory for user. Prentice Hall, Englewood CliffsMATH
17.
18.
Bengio Y, Lamblin P, Popovici D, Larochelle H (2007) Greedy layer-wise training of deep networks. In: Advances in neural information processing systems (NIPS’06). MIT Press, Cambridge, pp 153–160
19.
de la Rosa E, Yu W (2016) Randomized algorithms for nonlinear system identification with deep learning modification. Inform Sci 364:197–212CrossRef
20.
Narendra KS, Parthasarathy K (1990) Identification and control of dynamical systems using neural networks. IEEE Trans Neural Netw 1(1):4-2CrossRef
21.
Jagannathan S, Lewis FL (1996) Identification of nonlinear dynamical systems using multilayered neural networks. Automatica 32(12):1707–1712MathSciNetCrossRefMATH
22.
Busseti E, Osband I, Wong S (2012) Deep learning for time series modeling. CS 229 Technical Report, Stanford University
23.
de la Rosa E, Yu W (2015) Restricted Boltzmann machine for nonlinear system modeling. In: 14th IEEE international conference on machine learning and applications (ICMLA15), Miami, USA
24.
de la Rosa E, Yu W, Li X (2016) Nonlinear system modeling with deep neural networks and autoencoders algorithm. In: 2016 IEEE international conference on systems, man, and cybernetics (SMC16), Budapest, Hungary, pp 2157–2162
25.
de la Rosa E, Yu W (2015) Nonlinear system identification using deep learning and randomized algorithms. In: 2015 IEEE international conference on information and automation, Lijing, China, pp 274–279
26.
27.
Cybenko G (1989) Approximation by superposition of sigmoidal activation function. Math Control Sig Syst 2:303–314CrossRefMATH
28.
Huang GB, Chen L, Siew CK (2006) Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans Neural Netw. 17(4):879–92CrossRef
29.
Ackley DH, Hinton GE, Sejnowski TJ (1985) A learning algorithm for boltzmann machines. Cogn Sci 9:147–169CrossRef
30.
31.
Box G, Jenkins G, Reinsel G (2008) Time series analysis: forecasting and control, 4th edn. Wiley, New YorkCrossRefMATH
32.
Bergstra J, Bengio Y (2011) Random search for hyper-parameter optimization. J Machine Learn Res 13:281–305MathSciNetMATH
33.
Schoukens J, Suykens J, Ljung L (2009) Wiener-Hammerstein benchmark, 15th IFAC symposiumon system identification. Saint-Malo, France
34.
Erhan D, Manzagol PA, Bengio Y, Bengio S, Vincent P (2009) The difficulty of training deep architectures and the effect of unsupervised pretraining. In: 12th International conference on artificial intelligence and statistics (AISTATS’09), pp 153–160
35.
Bartlett PL (1997) For valid generalization, the size of the weights is more important than the size of the network. In: Mozer M, Jordan M, Petsche T (eds) Advances in neural information processing systems’ 1996, vol 9. MIT Press, Cambridge, pp 134–140
36.
Jang JS (1993) ANFIS: Adaptive-network-based fuzzy inference system. IEEE Trans Syst Man Cybern 23:665–685CrossRef
37.
Mitra S, Hayashi Y (2000) Neuro-fuzzy rule generation: survey in soft computing framework. IEEE Trans Neural Netw 11(3):748–769CrossRef
38.
Rivals I, Personnaz L (2003) Neural-network construction and selection in nonlinear modeling. IEEE Trans Neural Netw 14(4):804–820CrossRefMATH
39.
40.
Leung FHF, Lam HK, Ling SH, Tam PKS (2003) Tuning of the structure and parameters of a neural network using an improved genetic algorithm. IEEE Transact Neural Netw 14:79–88CrossRef
Metadata
Title
Deep Boltzmann machine for nonlinear system modelling
Authors
Wen Yu
Erick de la Rosa
Publication date
20-06-2018
Publisher
Springer Berlin Heidelberg
Published in
International Journal of Machine Learning and Cybernetics / Issue 7/2019
Print ISSN: 1868-8071
Electronic ISSN: 1868-808X
DOI
https://doi.org/10.1007/s13042-018-0847-0

Other articles of this Issue 7/2019

International Journal of Machine Learning and Cybernetics 7/2019 Go to the issue

Original Article

An emergency decision making method based on the multiplicative consistency of probabilistic linguistic preference relations

Original Article

Optimal pricing decision for supply chains with risk sensitivity and human estimation

Original Article

A voice activity detection algorithm in spectro-temporal domain using sparse representation

Original Article

Joint sparse representation and locality preserving projection for feature extraction

Original Article

A reinforced fuzzy ARTMAP model for data classification

Original Article

Weight-sharing multi-stage multi-scale ensemble convolutional neural network