Top

Neural Computing and Applications

Published in:

04-08-2020 | Original Article

Selecting data adaptive learner from multiple deep learners using Bayesian networks

Authors: Shusuke Kobayashi, Susumu Shirayama

Published in: Neural Computing and Applications | Issue 9/2021

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

A method to predict time series using multiple deep learners and a Bayesian network is proposed. In this study, the input explanatory variables are Bayesian network nodes that are associated with learners. Training data are divided using K-means clustering, and multiple deep learners are trained depending on the cluster. A Bayesian network is used to determine which deep learner is in charge of predicting a time series. We determine a threshold value and select learners with a posterior probability equal to or greater than the threshold value, which could facilitate more robust prediction. The proposed method is applied to financial time-series data, and the predicted results for the Nikkei 225 index are demonstrated.

previous article Source localization in resource-constrained sensor networks based on deep learning

next article Density-weighted support vector machines for binary class imbalance learning

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Bergstra J, Bengio Y (2012) Random search for hyperparameter optimization. J Mach Learn Res 13:281–305MathSciNetMATH

Loshchilov I, Hutter F (2016) CMA-ES for hyperparameter optimization of deep neural networks. CoRR

Lorenzo PR, Nalepa J, Kawulok M, Ramos LS, Pastor JR (2017) Particle swarm optimization for hyper- parameter selection in deep neural networks. In: Proceedings of the genetic and evolutionary computation conference. ACM, pp 481–488

Snoek J, Larochelle H, Adams RP (2012) Practical Bayesian optimization of machine learning algorithms. In: Advances in neural information processing systems, pp 2951–2959

Kuremoto T, Kimura S, Kobayashi K, Obayashi M (2014) Time-series forecasting using a deep belief network with restricted Boltzmann machines. Neurocomputing 137:47–56CrossRef

Bengio Y, Lamblin P, Popovici D, Larochelle H (2007) Greedy layer-wise training of deep networks. In: Advances in neural information processing systems, pp 153–160

Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105

Dahl GE, Yu D, Deng L, Acero A (2012) Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition. IEEE Transactions on audio, speech, and language processing 20(1):30–42CrossRef

Wang HZ, Li GQ, Wang GB, Peng JC, Jiang H, Liu YT (2017) Deep learning based ensemble approach for probabilistic wind power forecasting. Appl Energy 188:56–70CrossRef

10.

Suk HI, Lee SW, Shen D, Alzheimerer’s (2017) Disease neuroimaging initiative deep ensemble learning of sparse regression models for brain disease diagnosis. Med Image Anal 3:101–113CrossRef

11.

Zhao Y, Li J, Yu L (2017) A deep learning ensemble approach for crude oil price forecasting. Energy Econ 66:9–16CrossRef

12.

Takahashi Y, Asada M (1999) Behavior acquisition by multi-layered reinforcement learning. In: Proceedings of 1999 IEEE international conference on systems, man, and cybernetics, pp 716–721

13.

Jacob RA, Jordan MI, Nowlan SJ, Hinton GE (1991) Adaptive mixture of local experts. Neural Comput 3(1):79–87CrossRef

14.

Zhang H, Liu G, Chow TWS, Liu W (2011) Textual and visual content-based anti-phishing: a Bayesian approach. IEEE Trans Neural Netw 22(10):1532–1546CrossRef

15.

Kobayashi S, Shirayama S (2017) Time series forecasting with multiple deep learners: selection from a Bayesian network. J Data Anal Inf Process 5:115–130

16.

Nomiya H, Uehara K (2007) Multistrategical image classification for image data mining. In: Proceedings of international workshop on multimedia data mining, pp 22–30

17.

Takahashi Y, Takeda M, Asada M (1999) Continuous valued Q-learning for vision-guided behavior acquisition. In: Proceedings of 1999 IEEE/SICE/RSJ international conference on multisensor fusion and integration for intelligent systems, pp 255–260

18.

Collobert R, Bengio S, Bengio Y (2002) A parallel mixture of SVMs for very large scale problems. Neural Comput 14(5):1105–1114CrossRef

19.

Tresp V (2000) Mixture of Gaussian processes. In: Proceedings of the 13th international conference on neural information proceeding system, pp 633–639

20.

Theis L, Bethge M (2015) Generative image modeling using spatial LSTMs. In: Proceedings of the 28th international conference on neural information proceeding system, pp 1927–1935

21.

Deisenroth MP, Ng JW (2015) Distributed Gaussian processes. In: Proceedings of the 32nd international conference on international conference on machine learning, pp 1481–1490

22.

Shahbaba B, Neal R (2009) Nonlinear models using Dirichlet process mixtures. J Mach Learn Res 10:1829–1850MathSciNetMATH

23.

Eigen D, Ranzato MA, Sutskever I (2004) Learning factored representations in a deep mixture of experts. In: Workshop proceedings of the international conference on learning representations

24.

Shazeer N, Mirhoseini A, Maziarz K, Davis A, Le Q, Hinton G, Dean J (2017) Outrageously large neural networks: the sparsely-gated mixture-of-experts layer. In: Conference proceedings of the international conference on learning representations

25.

Gross S, Gross S, Ranzato M, Szlam A (2017) Hard mixtures of experts for large scale weakly supervised vision. In: 2017 IEEE conference on computer vision and pattern recognition, pp 5085–5093

26.

Pelleg D, Moore A (2000) X-means: extending K-means with efficient estimation of the number of clusters. In: Proceedings of 7th international conference on machine learning, pp 727—734

27.

Geiger D, Heckerman D (1994) Learning Gaussian networks. In: Tenth conference on uncertainty in artificial intelligence, pp 235–243

28.

Sak H, Senior A, Beaufays F (2014) Long short-term memory recurrent neural network architectures for large scale acoustic modeling. In: Proceedings of the annual conference of international speech communication association, pp 338–342

29.

Scutari M, Vitolo C, Tucker A (2019) Learning Bayesian networks from big data with greedy search: computational complexity and efficient implementation. Stat Comput 29:1095–1108MathSciNetCrossRef

30.

Scutari M (2010) Learning Bayesian networks with the bnlearn R Package. J Stat Softw 35(3):1–22CrossRef

Title: Selecting data adaptive learner from multiple deep learners using Bayesian networks
Authors: Shusuke Kobayashi
Susumu Shirayama
Publication date: 04-08-2020
Publisher: Springer London
Published in: Neural Computing and Applications / Issue 9/2021
Print ISSN: 0941-0643
Electronic ISSN: 1433-3058
DOI: https://doi.org/10.1007/s00521-020-05234-6

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Springer Professional "Wirtschaft+Technik"

Other articles of this Issue 9/2021

Overflow warning and remote monitoring technology based on improved random forest

Synergy evaluation model of container multimodal transport based on BP neural network

Application of the group method of data handling and variable importance analysis for prediction and modelling of saltwater intrusion processes in coastal aquifers

Next word prediction based on the N-gram model for Kurdish Sorani and Kurmanji

Exploring resource management for innovation power network based on deep learning algorithm

Domain-adaptive intelligence for fault diagnosis based on deep transfer learning from scientific test rigs to industrial applications

Premium Partner