Skip to main content
Fachgebiete
Automobil + Motoren Bauwesen + Immobilien Business IT + Informatik Elektrotechnik + Elektronik Energie + Nachhaltigkeit Finance + Banking Management + Führung Marketing + Vertrieb Maschinenbau + Werkstoffe Versicherung + Risiko
DE
EN
Bücher
Zeitschriften
Themenseiten
Organisationspsychologie Projektmanagement Marketing Smart Manufacturing
Weitere Formate
Podcasts Webinare Technik Webinare Wirtschaft Kongresse Veranstaltungskalender Awards
Jetzt Einzelzugang starten
Zugang für Unternehmen
Referenzkunden
SLX-Digitalkonferenz: Zukunftswerkstatt 2024

Mehr Umsatz mit Top-Kontakten

Am 7. Mai erfahren Sie, wie Vertriebsteams Leadgenerierung, Leadnurturing und Leadstrategien effizient steuern können.
Erweiterte Suche
Anmelden
nach oben
Soft Computing
Erschienen in: Soft Computing 4/2016

10.02.2015 | Methodologies and Application

Parallel implementation of multilayered neural networks based on Map-Reduce on cloud computing clusters

verfasst von: Hai-jun Zhang, Nan-feng Xiao

Erschienen in: Soft Computing | Ausgabe 4/2016

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

To meet the requirements of big data processing, this paper presents an efficient mapping scheme for a fully connected multilayered neural network, which is trained by using back-propagation (BP) algorithm based on Map-Reduce of cloud computing clusters. The batch-training (or epoch-training) regimes are used by effective segmentation of samples on the clusters, and are adopted in the separated training method, weight summary to achieve convergence by iterating. For a parallel BP algorithm on the clusters and a serial BP algorithm on an uniprocessor, the required time for implementing the algorithms is derived. The performance parameters, such as speedup, optimal number and minimum of data nodes are evaluated for the parallel BP algorithm on the clusters. Experiment results demonstrate that the proposed parallel BP algorithm in this paper has better speedup, faster convergence rate, less iterations than that of the existed algorithms.
Vorheriger Artikel Least squares support vector regression and interval type-2 fuzzy density weight for scene denoising
Nächster Artikel Knowledge representation using interval-valued fuzzy formal concept lattice

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Jetzt informieren
Literatur
Alham NK et al (2013) A MapReduce-based distributed SVM ensemble for scalable image classification and annotation. Comput Math Appl 66(10):1920–1934
Bengio Y, Lamblin P, Popovici D, Larochelle H (2006) Greedy layer-wise training of deep networks. In: Proceedings of advances in neural information processing systems, pp 153–160
Chen G (2005) Parallel computing architecture, algorithm, programming [M]. Higher Education Press, Beijing, pp 1–366
Chu LC, Wah BW (1992) Optimal mapping of neural network learning on message-passing multicomputers. J Parallel Distrib Comput 14:319–339CrossRef
Dean J, Ghemawat S (2008) Mapreduce: simplified data processing on large clusters. In: OSDI ’04, pp 137–150
Deprit E (1989) Implementing recurrent backpropagation on the connection machine. Neural Netw 2:295–314CrossRef
El-Amawy A, Kulasinghe P (1997) Algorithmic mapping of feedforward neural networks onto multiple bus systems. IEEE Trans Parallel Distrib Syst 8(2):130–136CrossRef
Erhan D, Manzagol P, Bengio Y, Bengio S, Vincent P (2009) The difficulty of training deep architectures and the effect of unsupervised pre-training. In: Proceedings of the international conference on artificial intelligence and statistics, pp 153–160
Eskil MT, Benli KS (2014) Facial expression recognition based on anatomy. Comput Vis Image Underst 119:1–14CrossRef
Fahlman SE, Hinton GE (1987) Connectionist architectures for artificial intelligence. IEEE Comput 20:100–109CrossRef
Feldman JA et al (1988) Computing with structured connectionist networks. Commun ACM 31(2):170–187CrossRef
Fischer Asja, Igel C (2014) Training restricted Boltzmann machines: an introduction. Pattern Recognit 47:25–39CrossRefMATH
Foo SK, Saratchandran P, Sundararajan N (1995) Application of genetic algorithm for parallel implementation of backpropagation neural networks. In: Proceedings of the international symposium on intelligent robotic systems, pp 76–79
Han L (2007) Artificial neural networks tutorial [M]. Beijing University of Posts and Telecommunications Press of China, Beijing
Haykin S, Deng C (1991) Classification of radar clutter using neural networks. IEEE Trans Neural Netw 2:589–600CrossRef
Haykins S (1999) Neural networks–a comprehensive foundation. Prentice Hall Int’l, Englewood Cliffs
Heo H, Park H, Kim N, Lee J (2009) Prediction of credit delinquents using locally transductive multi-layer perceptron. Neurocomputing 73:169–175CrossRef
Isa NAM, Mamat WMFW (2011) Clustered-hybrid multilayer perceptron network for pattern recognition application. Appl Soft Comput 11:1457–1466CrossRef
Jin X, Yang G, Yin Y, Man H, He H (2014) Sparse-representation-based classification with structure-preserving dimension reduction. Cognit Comput 6(3):608–621CrossRef
Khan ER (1991) Systonlic architectures for artificial neural nets. In: IEEE international joint conference on neural networks, vol 1, pp 620–627, 18–21 Nov 1991
Kosko B (ed) (1992) Neural networks for signal processing. Prentice Hall, Englewood Cliffs
Kumar V, Shekhar S, Amin MB (1994) A scalable parallel formulation of the backpropagation algorithm for hypercubes and related architectures. IEEE Trans Parallel Distrib Syst 5:1073–1090CrossRef
Kumar V, Shekhar S, Amin MB (1994) A scalable parallel formulation of the backpropagation algorithm for hypercubes and related architectures. IEEE Trans Parallel Distrib Syst 5:1073–1090CrossRef
Leighton T (1992) Introduction to parallel algorithms and architectures. Morgan Kaufmann Publishers, San MateoMATH
Li H, Li H, Wei Y, Tang Y, Wang Q (2014) Sparse-based neural response for image classification. Neurocomputing 144:198–207CrossRef
Liew WS, Seera M, Loo CK, Lim E (2015) Affect classification using genetic-optimized ensembles of fuzzyARTMAPs. Appl Soft Comput 27:53–63CrossRef
Lin Wenhui, Liu Jun (2013) Performance analysis of map-reduce program in heterogeneous cloud computing. J Netw 8(8):1734–1741
Luttrell SP (1989) Image compression using a multilayer neural network. Pattern Recognit 10:1–7CrossRefMATH
Madhyastha TM, Reed DA (2002) Learning to classify parallel input/output access patterns. IEEE Trans Parallel Distrib Syst 13(8):802–813CrossRef
Malalur SS, Manry MT, Jesudhas P (2015) Multiple optimal learning factors for the multi-layer perceptron. Neurocomputing 149:1490–1501CrossRef
Malkin J, Subramanya A, Bilmes J (2009) On the semi-supervised learning of multi-layered perceptrons. In: Proceedings of the annual conference of the International Speech Communication Association, Brighton, UK, September 2009
Markowska-Kaczmar U, Koldowski M (2014) Spiking neural network vs multilayer perceptron: who is the winner in the racing car computer game. Soft Comput. doi:10.​1007/​s00500-014-1515-2
Matei Zaharia et al (2012) Resilient distributed datasets: a fault-tolerant abstraction for in-memory cluster computing. In: Proceedings of the 9th USENIX conference on networked systems design and implementation, USENIX association
Murphey YL, Luo Y (2002) Feature extraction for a multiple pattern classification neural network system. Pattern Recognit Proc. 2:220–223
Nikoonahad M, Liu DC (1990a) Medical ultrasound imaging using neural networks. Electron Lett 26:545–546
Nikoonahad M, Liu DC (1990b) Medical ultra sound imaging using neural networks. Electron Lett 2(6):18–23
Rumelhart DE, McClelland JL (eds) (1986a) Parallel and distributed processing (PDP): explorations in the microstructure of the cognition. MIT Press, Cambridge
Rumelhart DE, McClelland JL (eds) (1986b) Explorations in the micro structure of the cognition, parallel and distributed processing. MIT Press, Cambridge
Schmidhuber J (2015) Deep learning in neural networks: an overview. Neural Netw 61:85–117CrossRef
Sejnowski TJ, Rosenberg CR (1987) Parallel networks that learn to pronounce English text. Complex Syst 1:145–168MATH
Shams S, Przytula KW (1991) Implementation of multilayer neural networks on parallel programmable digital computers. In: Bayoumi M (ed) Parallel algorithms and architectures for DSP applications. Kluwer, Norwell, pp 225–253CrossRef
Sudhakar V, Siva C, Murthy R (1998) Efficient mapping of back-propagation algorithm onto a network of workstations. IEEE Trans Man Mach Cybern Part B Cybern 28(6):841–848CrossRef
Suk H-I, Lee S-W, Shen D (2014) Hierarchical feature representation and multimodal fusion with deeplearning for AD/MCI diagnosis. NeuroImage 101:569–582CrossRef
Sundaram V (1990) PVM: a framework for parallel and distributed computing. Concurr Pract Exp 12:315–319CrossRef
Sundararajan N, Saratchandran P (1998) Parallel architecture for artificial neural networks: paradigms and implementations. IEEE CS Press, Los Alamitos
Suresh S, Omkar SN, Mani V (2005) Parallel implementation of back-propagation algorithm in networks of workstations. IEEE Trans Parallel Distrib Syst 16(1):24–34CrossRef
Torija AJ, Ruiz DP (2015) A general procedure to generate models for urban environmental-noise pollution using feature selection and machine learning methods. Sci Total Environ 505:680–693CrossRef
Triguero I, Peralta D, Bacardit J, García S, Herrera F (2015) MRPR: a MapReduce solution for prototype reduction in big data classification. Neurocomputing 150:331–345CrossRef
Xu J, Man H (2011) Dictionary learning based on Laplacian score in sparse coding. Lect Notes Comput Sci 6871:253–264CrossRef
Xu J, He H, Man H (2012a) DCPE co-training for classification. Neurocomputing 86:75–85
Xu J, Yin Y, Man H (2012b) Feature selection based on sparse imputation. In: The 2012 international joint conference on neural networks (IJCNN), pp 1–7
Yoon H, Nang JH (1990) Multilayer neural networks on distributed memory multiprocessors. In: Proceedings of the international neural network conference, Paris, France
Zeng X, Luo S, Li Q (2010) An associative sparse coding neural network and applications. Neurocomputing 73:684–9CrossRef
Zhou ZH, Li M (2010) Semi-supervised learning by disagreement. Knowl Inf Syst 24(3):415–439CrossRef
Zhu C, Rao R (2012) The improved BP algorithm based on MapReduce and genetic algorithm. In: IEEE international conference on computer science and service system, pp 1567–1570
Metadaten
Titel
Parallel implementation of multilayered neural networks based on Map-Reduce on cloud computing clusters
verfasst von
Hai-jun Zhang
Nan-feng Xiao
Publikationsdatum
10.02.2015
Verlag
Springer Berlin Heidelberg
Erschienen in
Soft Computing / Ausgabe 4/2016
Print ISSN: 1432-7643
Elektronische ISSN: 1433-7479
DOI
https://doi.org/10.1007/s00500-015-1599-3

Weitere Artikel der Ausgabe 4/2016

Soft Computing 4/2016 Zur Ausgabe

Methodologies and Application

A particle swarm inspired cuckoo search algorithm for real parameter optimization

Methodologies and Application

Using a branch-and-bound and a genetic algorithm for a single-machine total late work scheduling problem

Methodologies and Application

Modeling and implementation of Z-number

Methodologies and Application

An enhanced model for the integrated production and transportation problem in a multiple vehicles environment

Methodologies and Application

Co-evolution-based immune clonal algorithm for clustering

Methodologies and Application

A modified probability collectives optimization algorithm based on trust region method and a new temperature annealing schedule