Skip to main content
Top

2021 | OriginalPaper | Chapter

The Streaming Approach to Training Restricted Boltzmann Machines

Authors : Piotr Duda, Leszek Rutkowski, Piotr Woldan, Patryk Najgebauer

Published in: Artificial Intelligence and Soft Computing

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

One of the greatest challenges facing researchers of machine learning algorithms nowadays is the desire to minimize the training time of these algorithms. One of the most promising and unexplored structures of the neural network is the Restricted Boltzmann Machine. In this paper, we propose to use the BBTADD algorithm for RBM training. The performance of the algorithm has been illustrated on one of the most popular data sets.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
2.
go back to reference Bilski, J., Kowalczyk, B., Marchlewska, A., Zurada, J.M.: Local Levenberg-Marquardt algorithm for learning feedforwad neural networks. J. Artif. Intell. Soft Comput. Res. 10(4), 299–316 (2020)CrossRef Bilski, J., Kowalczyk, B., Marchlewska, A., Zurada, J.M.: Local Levenberg-Marquardt algorithm for learning feedforwad neural networks. J. Artif. Intell. Soft Comput. Res. 10(4), 299–316 (2020)CrossRef
3.
go back to reference Bodyanskiy, Y., Vynokurova, O., Pliss, I., Setlak, G., Mulesa, P.: Fast learning algorithm for deep evolving GMDH-SVM neural network in data stream mining tasks. In: 2016 IEEE First International Conference on Data Stream Mining Processing (DSMP), pp. 257–262, August 2016 Bodyanskiy, Y., Vynokurova, O., Pliss, I., Setlak, G., Mulesa, P.: Fast learning algorithm for deep evolving GMDH-SVM neural network in data stream mining tasks. In: 2016 IEEE First International Conference on Data Stream Mining Processing (DSMP), pp. 257–262, August 2016
4.
go back to reference deBarros, R.S.M., Hidalgo, J.I.G., de Lima Cabral, D.R.: Wilcoxon rank sum test drift detector. Neurocomputing 275, 1954–1963 (2018) deBarros, R.S.M., Hidalgo, J.I.G., de Lima Cabral, D.R.: Wilcoxon rank sum test drift detector. Neurocomputing 275, 1954–1963 (2018)
5.
go back to reference Duda, P., Jaworski, M., Cader, A., Wang, L.: On training deep neural networks using a streaming approach. J. Artif. Intell. Soft Comput. Res. 10(1), 15–26 (2020)CrossRef Duda, P., Jaworski, M., Cader, A., Wang, L.: On training deep neural networks using a streaming approach. J. Artif. Intell. Soft Comput. Res. 10(1), 15–26 (2020)CrossRef
6.
go back to reference Duda, P., Przybyszewski, K., Wang, L.: A novel drift detection algorithm based on features’ importance analysis in a data streams environment. J. Artif. Intell. Soft Comput. Res. 10, 287–298 (2020) Duda, P., Przybyszewski, K., Wang, L.: A novel drift detection algorithm based on features’ importance analysis in a data streams environment. J. Artif. Intell. Soft Comput. Res. 10, 287–298 (2020)
8.
go back to reference Elsaeidy, A., Munasinghe, K.S., Sharma, D., Jamalipour, A.: Intrusion detection in smart cities using restricted Boltzmann machines. J. Netw. Comput. Appl. 135, 76–83 (2019)CrossRef Elsaeidy, A., Munasinghe, K.S., Sharma, D., Jamalipour, A.: Intrusion detection in smart cities using restricted Boltzmann machines. J. Netw. Comput. Appl. 135, 76–83 (2019)CrossRef
9.
go back to reference Gałlowski, T., Krzyzak, A., Filutowicz, Z.: A new approach to detection of changes in multidimensional patterns. J. Artif. Intell. Soft Comput. Res. 10(2), 125–136 (2020)CrossRef Gałlowski, T., Krzyzak, A., Filutowicz, Z.: A new approach to detection of changes in multidimensional patterns. J. Artif. Intell. Soft Comput. Res. 10(2), 125–136 (2020)CrossRef
11.
go back to reference Gantz, J., Reinsel, D.: The digital universe in 2020: Big data, bigger digital shadows, and biggest growth in the far east. IDC iView: IDC Analyze Future 2007(2012), 1–16 (2012) Gantz, J., Reinsel, D.: The digital universe in 2020: Big data, bigger digital shadows, and biggest growth in the far east. IDC iView: IDC Analyze Future 2007(2012), 1–16 (2012)
12.
go back to reference Hinton, G.E.: Training products of experts by minimizing contrastive divergence. Neural Comput. 14(8), 1771–1800 (2002)CrossRef Hinton, G.E.: Training products of experts by minimizing contrastive divergence. Neural Comput. 14(8), 1771–1800 (2002)CrossRef
14.
go back to reference Jaworski, M., Duda, P., Rutkowski, L.: On applying the restricted Boltzmann machine to active concept drift detection. In: 2017 IEEE Symposium Series on Computational Intelligence (SSCI), pp. 1–8. IEEE (2017) Jaworski, M., Duda, P., Rutkowski, L.: On applying the restricted Boltzmann machine to active concept drift detection. In: 2017 IEEE Symposium Series on Computational Intelligence (SSCI), pp. 1–8. IEEE (2017)
15.
go back to reference Jaworski, M., Duda, P., Rutkowski, L.: Concept drift detection in streams of labelled data using the restricted Boltzmann machine. In: 2018 International Joint Conference on Neural Networks (IJCNN), pp. 1–7. IEEE (2018) Jaworski, M., Duda, P., Rutkowski, L.: Concept drift detection in streams of labelled data using the restricted Boltzmann machine. In: 2018 International Joint Conference on Neural Networks (IJCNN), pp. 1–7. IEEE (2018)
16.
go back to reference Jaworski, M., Rutkowski, L., Angelov, P.: Concept drift detection using autoencoders in data streams processing. In: Rutkowski, L., Scherer, R., Korytkowski, M., Pedrycz, W., Tadeusiewicz, R., Zurada, J.M. (eds.) ICAISC 2020. LNCS (LNAI), vol. 12415, pp. 124–133. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-61401-0_12CrossRef Jaworski, M., Rutkowski, L., Angelov, P.: Concept drift detection using autoencoders in data streams processing. In: Rutkowski, L., Scherer, R., Korytkowski, M., Pedrycz, W., Tadeusiewicz, R., Zurada, J.M. (eds.) ICAISC 2020. LNCS (LNAI), vol. 12415, pp. 124–133. Springer, Cham (2020). https://​doi.​org/​10.​1007/​978-3-030-61401-0_​12CrossRef
17.
go back to reference Jaworski, M., Rutkowski, L., Duda, P., Cader, A.: Resource-aware data stream mining using the restricted Boltzmann machine. In: Rutkowski, L., Scherer, R., Korytkowski, M., Pedrycz, W., Tadeusiewicz, R., Zurada, J.M. (eds.) ICAISC 2019. LNCS (LNAI), vol. 11509, pp. 384–396. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-20915-5_35CrossRef Jaworski, M., Rutkowski, L., Duda, P., Cader, A.: Resource-aware data stream mining using the restricted Boltzmann machine. In: Rutkowski, L., Scherer, R., Korytkowski, M., Pedrycz, W., Tadeusiewicz, R., Zurada, J.M. (eds.) ICAISC 2019. LNCS (LNAI), vol. 11509, pp. 384–396. Springer, Cham (2019). https://​doi.​org/​10.​1007/​978-3-030-20915-5_​35CrossRef
18.
go back to reference Krizhevsky, A., Hinton, G.: Convolutional deep belief networks on CIFAR-10. Unpublished Manuscript 40(7), 1–9 (2010) Krizhevsky, A., Hinton, G.: Convolutional deep belief networks on CIFAR-10. Unpublished Manuscript 40(7), 1–9 (2010)
19.
go back to reference Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. Adv. Neural. Inf. Process. Syst. 25, 1097–1105 (2012) Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. Adv. Neural. Inf. Process. Syst. 25, 1097–1105 (2012)
20.
go back to reference Kumar, R., Weill, E., Aghdasi, F., Sriram, P.: A strong and efficient baseline for vehicle re-identification using deep triplet embedding. J. Artif. Intell. Soft Comput. Res. 10(1), 27–45 (2019)CrossRef Kumar, R., Weill, E., Aghdasi, F., Sriram, P.: A strong and efficient baseline for vehicle re-identification using deep triplet embedding. J. Artif. Intell. Soft Comput. Res. 10(1), 27–45 (2019)CrossRef
21.
go back to reference Li, C., Wang, J., Ye, X.: Using a recurrent neural network and restricted Boltzmann machines for malicious traffic detection. NeuroQuantology 16(5) (2018) Li, C., Wang, J., Ye, X.: Using a recurrent neural network and restricted Boltzmann machines for malicious traffic detection. NeuroQuantology 16(5) (2018)
22.
go back to reference Mahdi, O.A., Pardede, E., Ali, N., Cao, J.: Diversity measure as a new drift detection method in data streaming. Knowl.-Based Syst. 191, 105227 (2020) Mahdi, O.A., Pardede, E., Ali, N., Cao, J.: Diversity measure as a new drift detection method in data streaming. Knowl.-Based Syst. 191, 105227 (2020)
23.
go back to reference Melko, R.G., Carleo, G., Carrasquilla, J., Cirac, J.I.: Restricted Boltzmann machines in quantum physics. Nat. Phys. 15(9), 887–892 (2019)CrossRef Melko, R.G., Carleo, G., Carrasquilla, J., Cirac, J.I.: Restricted Boltzmann machines in quantum physics. Nat. Phys. 15(9), 887–892 (2019)CrossRef
24.
go back to reference Niksa-Rynkiewicz, T., Szewczuk-Krypa, N., Witkowska, A., Cpalka, K., Zalasinski, M., Cader, A.: Monitoring regenerative heat exchanger in steam power plant by making use of the recurrent neural network. J. Artif. Intell. Soft Comput. Res. 11(2), 143–155 (2021) Niksa-Rynkiewicz, T., Szewczuk-Krypa, N., Witkowska, A., Cpalka, K., Zalasinski, M., Cader, A.: Monitoring regenerative heat exchanger in steam power plant by making use of the recurrent neural network. J. Artif. Intell. Soft Comput. Res. 11(2), 143–155 (2021)
26.
go back to reference Passos, L.A., Papa, J.P.: Temperature-based deep Boltzmann machines. Neural Process. Lett. 48(1), 95–107 (2018)CrossRef Passos, L.A., Papa, J.P.: Temperature-based deep Boltzmann machines. Neural Process. Lett. 48(1), 95–107 (2018)CrossRef
27.
go back to reference Rahman, J.S., Gedeon, T., Caldwell, S., Jones, R., Jin, Z.: Towards effective music therapy for mental health care using machine learning tools: Human affective reasoning and music genres. J. Artif. Intell. Soft Comput. Res. 11(1), 5–20 (2020)CrossRef Rahman, J.S., Gedeon, T., Caldwell, S., Jones, R., Jin, Z.: Towards effective music therapy for mental health care using machine learning tools: Human affective reasoning and music genres. J. Artif. Intell. Soft Comput. Res. 11(1), 5–20 (2020)CrossRef
28.
go back to reference Read, J., Perez-Cruz, F., Bifet, A.: Deep learning in partially-labeled data streams. In: Proceedings of the 30th Annual ACM Symposium on Applied Computing, SAC 2015, pp. 954–959. ACM, New York, NY, USA (2015) Read, J., Perez-Cruz, F., Bifet, A.: Deep learning in partially-labeled data streams. In: Proceedings of the 30th Annual ACM Symposium on Applied Computing, SAC 2015, pp. 954–959. ACM, New York, NY, USA (2015)
30.
go back to reference Salakhutdinov, R., Mnih, A., Hinton, G.: Restricted Boltzmann machines for collaborative filtering. In: Proceedings of the 24th International Conference on Machine Learning, pp. 791–798 (2007) Salakhutdinov, R., Mnih, A., Hinton, G.: Restricted Boltzmann machines for collaborative filtering. In: Proceedings of the 24th International Conference on Machine Learning, pp. 791–798 (2007)
31.
go back to reference Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014) Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:​1409.​1556 (2014)
32.
go back to reference Smolensky, P.: Information processing in dynamical systems: Foundations of harmony theory. Technical report, Colorado University at Boulder Department of Computer Science (1986) Smolensky, P.: Information processing in dynamical systems: Foundations of harmony theory. Technical report, Colorado University at Boulder Department of Computer Science (1986)
33.
go back to reference Zhang, N., Ding, S., Zhang, J., Xue, Y.: An overview on restricted Boltzmann machines. Neurocomputing 275, 1186–1199 (2018)CrossRef Zhang, N., Ding, S., Zhang, J., Xue, Y.: An overview on restricted Boltzmann machines. Neurocomputing 275, 1186–1199 (2018)CrossRef
34.
go back to reference Zini, J.E., Rizk, Y., Awad, M.: An optimized parallel implementation of non-iteratively trained recurrent neural networks. J. Artif. Intell. Soft Comput. Res. 11(1), 33–50 (2020)CrossRef Zini, J.E., Rizk, Y., Awad, M.: An optimized parallel implementation of non-iteratively trained recurrent neural networks. J. Artif. Intell. Soft Comput. Res. 11(1), 33–50 (2020)CrossRef
Metadata
Title
The Streaming Approach to Training Restricted Boltzmann Machines
Authors
Piotr Duda
Leszek Rutkowski
Piotr Woldan
Patryk Najgebauer
Copyright Year
2021
DOI
https://doi.org/10.1007/978-3-030-87986-0_27

Premium Partner