Top

Neural Computing and Applications

Published in:

13-07-2017 | Original Article

Distributed cooperative learning algorithms using wavelet neural network

Authors: Jin Xie, Weisheng Chen, Hao Dai

Published in: Neural Computing and Applications | Issue 4/2019

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

This paper investigates the distributed cooperative learning (DCL) problems over networks, where each node only has access to its own data generated by the unknown pattern (map or function) uniformly, and all nodes cooperatively learn the pattern by exchanging local information with their neighboring nodes. These problems cannot be solved by using traditional centralized algorithms. To solve these problems, two novel DCL algorithms using wavelet neural networks are proposed, including continuous-time DCL (CT-DCL) algorithm and discrete-time DCL (DT-DCL) algorithm. Combining the characteristics of neural networks with the properties of the wavelet approximation, the wavelet series are used to approximate the unknown pattern. The DCL algorithms are used to train the optimal weight coefficient matrix of wavelet series. Moreover, the convergence of the proposed algorithms is guaranteed by using the Lyapunov method. Compared with existing distributed optimization strategies such as distributed average consensus (DAC) and alternating direction method of multipliers (ADMM), our DT-DCL algorithm requires less information communications and training time than ADMM strategy. In addition, it achieves higher accuracy than DAC strategy when the network consists of large amounts of nodes. Moreover, the proposed CT-DCL algorithm using a proper step size is more accurate than the DT-DCL algorithm if the training time is not considered. Several illustrative examples are presented to show the efficiencies and advantages of the proposed algorithms.

previous article Chaotic multi-verse optimizer-based feature selection

next article Correlation measure of hesitant fuzzy soft sets and their application in decision making

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Available only for authorised users

Chen W, Hua S, Ge SS (2014) Consensus-based distributed cooperative learning control for a group of discrete-time nonlinear multi-agent systems using neural networks. Automatica 50(9):2254–2268MathSciNetMATHCrossRef

Boyd S, Parikh N, Chu E et al (2011) Distributed optimization and statistical learning via the alternating direction method of multipliers. Found Trends Mach Learn 3(1):1–122MATHCrossRef

Li J, Lin X, Rui X et al (2014) A distributed approach toward discriminative distance metric learning. IEEE Trans Neural Netw Learn Syst 26(9):2111–2122MathSciNetCrossRef

Tekin C, Van der SM (2015) Distributed online learning via cooperative contextual bandits. Signal Process IEEE Trans 63(14):3700–3714MathSciNetMATHCrossRef

Chen T, Wang C, Hill DJ (2014) Rapid oscillation fault detection and isolation for distributed systems via deterministic learning. IEEE Trans Neural Netw Learn Syst 25(6):1187–1199CrossRef

Mertikopoulos P, Belmega EV, Moustakas AL et al (2012) Distributed learning policies for power allocation in multiple access channels. Onomr 30(1):96–106

Predd JB, Kulkarni SR, Vincent PH (2005) Distributed learning in wireless sensor networks. IEEE Signal Process Mag 23(4):56–69CrossRef

Scardapane S, Wang D, Panella M (2016) A decentralized training algorithm for echo state networks in distributed big data applications. Neural Netw 78:65–74MATHCrossRef

Georgopoulos L, Hasler M (2014) Distributed machine learning in networks by consensus. Neurocomputing 124(2):2–12CrossRef

10.

Chen J, Sayed AH (2011) Diffusion adaptation strategies for distributed optimization and learning over networks. IEEE Trans Signal Process 60(8):4289–4305MathSciNetMATHCrossRef

11.

Chen W, Hua S, Zhang H (2015) Consensus-based distributed cooperative learning from closed-loop neural control systems. IEEE Trans Neural Netw Learn Syst 26(2):331–345MathSciNetCrossRef

12.

Ai W, Chen W, Xie J (2016) A zero-gradient-sum algorithm for distributed cooperative learning using a feedforward neural network with random weights. Inf Sci 373:404–418CrossRef

13.

Ai W, Chen W, Xie J (2016) Distributed learning for feedforward neural networks with random weights using an event-triggered communication scheme. Neurocomputing 224:184–194CrossRef

14.

Olfati-Saber R, Fax JA, Murray RM (2007) Consensus and cooperation in networked multi-agent systems. Proc IEEE 95(1):215–233MATHCrossRef

15.

Xiao L, Boyd S, Kim SJ (2007) Distributed average consensus with least-mean-square deviation. J Parallel Distrib Comput 67(1):33–46MATHCrossRef

16.

Aysal T, Coates M, Rabbat M (2008) Distributed average consensus with dithered quantization. IEEE Trans Signal Process 56(10):4905–4918MathSciNetMATHCrossRef

17.

Lim C, Lee S, Choi JH et al (2014) Efficient implementation of statistical model-based voice activity detection using Taylor series approximation. IEICE Trans Fundam Electron Commun Comput Sci 97(3):865–868CrossRef

18.

Sharapudinov II (2014) Approximation of functions in variable-exponent Lebesgue and Sobolev spaces by finite Fourier–Haar series. Russ Acad Sci Sb Math 205(205):145–160MathSciNetMATH

19.

Huang GB, Saratchandran P, Sundararajan N (2005) A generalized growing and pruning RBF (GGAP-RBF) neural network for function approximation. IEEE Trans Neural Netw 16(1):57–67CrossRef

20.

Yang C, Jiang K et al (2017) Neural control of bimanual robots with guaranteed global stability and motion precision. IEEE Trans Ind Inf 13(3):1162–1171CrossRef

21.

Yang C, Wang X et al (2017) Neural-learning based telerobot control with guaranteed performance. IEEE Trans Cybern (in press). doi:10.1109/TCYB.2016.2573837

22.

Cui R, Yang C et al (2017) Adaptive neural network control of AUVs with control input nonlinearities using reinforcement learning. IEEE Trans Syst Man Cybern Syst 47(6):1019–1029CrossRef

23.

Wu S, Er MJ (2000) Dynamic fuzzy neural networks-a novel approach to function approximation. IEEE Trans Syst Man Cybern Part B Cybern A Publ IEEE Syst Man Cybern Soc 30(2):358–364CrossRef

24.

Ferrari S, Stengel RF (2005) Smooth function approximation using neural networks. IEEE Trans Neural Netw 16(1):24–38CrossRef

25.

Yang C, Yi Z, Zuo L (2008) Function approximation based on twin support vector machines. In: IEEE conference on cybernetics and intelligent systems, pp 259–264

26.

Zhang Q, Benveniste A (1991) Approximation by nonlinear wavelet networks. In: International conference on acoustics, speech and signal processing. ICASSP-91. pp 3417–3420

27.

Zhang Q, Benveniste A (1992) Wavelet networks. IEEE Trans Neural Netw 3(6):889CrossRef

28.

Delyon B, Juditsky A, Benveniste A (1995) Accuracy analysis for wavelet approximations. IEEE Trans Neural Netw 6(2):332–348CrossRef

29.

Zainuddin Z, Pauline O (2011) Modified wavelet neural network in function approximation and its application in prediction of time-series pollution data. Appl Soft Comput 11(8):4866–4874CrossRef

30.

Zainuddin Z, Ong P (2013) Design of wavelet neural networks based on symmetry fuzzy C-means for function approximation. Neural Comput Appl 23(1):247–259CrossRef

31.

Zainuddin Z, Ong P (2016) Optimization of wavelet neural networks with the firefly algorithm for approximation problems. Neural Comput Appl 28(7):1715–1728CrossRef

32.

Hou M, Han X, Gan Y (2009) Constructive approximation to real function by wavelet neural networks. Neural Comput Appl 18(8):883–889CrossRef

33.

Oysal Y, Yilmaz S (2010) An adaptive wavelet network for function learning. Neural Comput Appl 19(3):383–392CrossRef

34.

Xu J, Yan R (2011) Adaptive learning control for finite interval tracking based on constructive function approximation and wavelet. IEEE Trans Neural Netw 22(6):893–905CrossRef

35.

Alexandridis AK, Zapranis AD (2013) Wavelet neural networks: a practical guide. Neural Netw 42(1):1–27MATHCrossRef

36.

Yang C, Wang X et al (2016) Teleoperation control based on combination of wave variable and neural networks. IEEE Trans Syst Man Cybern Syst (in press). doi:10.1109/TSMC.2016.2615061

37.

Chen S, Zhao H, Zhang S et al (2013) Study of ultra-wideband fuze signal processing method based on wavelet transform. IET Radar Sonar Navig 8(3):167–172CrossRef

38.

Courroux S, Chevobbe S, Darouich M et al (2013) Use of wavelet for image processing in smart cameras with low hardware resources. J Syst Archit 59(10):826–832CrossRef

39.

Pavez E, Silva JF (2012) Analysis and design of wavelet-packet cepstral coefficients for automatic speech recognition. Speech Commun 54(6):814–835CrossRef

40.

Siddiqi MH, Lee SW, Khan AM (2014) Weed image classification using wavelet transform, stepwise linear discriminant analysis and support vector machines for an automatic spray control system. J Inf Sci Eng 30(4):1227–1244

41.

Yan R, Gao RX, Chen X (2014) Wavelets for fault diagnosis of rotary machines: a review with applications. Signal Process 96(5):1–15CrossRef

42.

Ganjefar S, Tofighi M (2015) Single-hidden-layer fuzzy recurrent wavelet neural network: applications to function approximation and system identification. Inf Sci 294:269–285MathSciNetMATHCrossRef

43.

Bazaraa MS, Goode JJ (1973) On symmetric duality in nonlinear programming. Oper Res 21(21):1–9MathSciNetMATHCrossRef

44.

Lu J, Tang CY (2011) Zero-gradient-sum algorithms for distributed convex optimization: the continuous-time case. IEEE Trans Autom Control 57(9):5474–5479MathSciNet

Title: Distributed cooperative learning algorithms using wavelet neural network
Authors: Jin Xie
Weisheng Chen
Hao Dai
Publication date: 13-07-2017
Publisher: Springer London
Published in: Neural Computing and Applications / Issue 4/2019
Print ISSN: 0941-0643
Electronic ISSN: 1433-3058
DOI: https://doi.org/10.1007/s00521-017-3134-1

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Springer Professional "Wirtschaft+Technik"

Other articles of this Issue 4/2019

Chaotic multi-verse optimizer-based feature selection

Automatic breast tumor detection in ABVS images based on convolutional neural network and superpixel patterns

Decision-making tool for crop selection for agriculture development

MR-SAS and electric power steering variable universe fuzzy PID integrated control

Modified multiple generalized regression neural network models using fuzzy C-means with principal component analysis for noise prediction of offshore platform

Adaptive pedestrian detection by predicting classifier

Premium Partner