Skip to main content
Top
Published in: Neural Computing and Applications 2/2023

04-10-2022 | Original Article

An improved parameter learning methodology for RVFL based on pseudoinverse learners

Authors: Xiaoxuan Sun, Xiaodan Deng, Qian Yin, Ping Guo

Published in: Neural Computing and Applications | Issue 2/2023

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

As a compact and effective learning model, the random vector functional link neural network (RVFL) has been confirmed with universal approximation capabilities. It has gained considerable attention in various fields. However, the randomly generated parameters in RVFL often lead to the loss of valid information and data redundancy, which severely degrades the model performance in practice. This paper first proposes an efficient network parameters learning approach for the original RVFL with pseudoinverse learner (RVFL-PL). Instead of taking the random feature mapping directly, RVFL-PL adopts a non-iterative manner to obtain influential enhancement nodes implanted with valuable information from input data, which realizes to improve the quality of the enhancement nodes and ease the problem caused by the randomly assigned parameters in the standard RVFL. Since the network parameters are optimized analytically, this improved variant can maintain the efficiency of the standard RVFL. Further, the RVFL-PL is extended to a multilayered structure (mRVFL-PL) to obtain high-level representations from the input data. The results of comprehensive experiments on some benchmarks indicate the performance improvement of the proposed method compared to other corresponding methods.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Yue K, Xu F, Yu J (2019) Shallow and wide fractional max-pooling network for image classification. Neural Comput Appl 31(2):409–419CrossRef Yue K, Xu F, Yu J (2019) Shallow and wide fractional max-pooling network for image classification. Neural Comput Appl 31(2):409–419CrossRef
2.
go back to reference Jia Y, Chen X, Yu J, Wang L, Wang Y (2021) Speaker recognition based on characteristic spectrograms and an improved self-organizing feature map neural network. Complex Intell Syst 7:1749–1757CrossRef Jia Y, Chen X, Yu J, Wang L, Wang Y (2021) Speaker recognition based on characteristic spectrograms and an improved self-organizing feature map neural network. Complex Intell Syst 7:1749–1757CrossRef
5.
go back to reference Cao W, Wang X, Ming Z, Gao J (2018) A review on neural networks with random weights. Neurocomputing 275:278–287CrossRef Cao W, Wang X, Ming Z, Gao J (2018) A review on neural networks with random weights. Neurocomputing 275:278–287CrossRef
6.
go back to reference Zhang L, Suganthan PN (2016) A survey of randomized algorithms for training neural networks. Inf Sci 364:146–155CrossRefMATH Zhang L, Suganthan PN (2016) A survey of randomized algorithms for training neural networks. Inf Sci 364:146–155CrossRefMATH
7.
go back to reference Pao Y, Park GH, Sobajic DJ (1994) Learning and generalization characteristics of the random vector functional-link net. Neurocomputing 6(2):163–180CrossRef Pao Y, Park GH, Sobajic DJ (1994) Learning and generalization characteristics of the random vector functional-link net. Neurocomputing 6(2):163–180CrossRef
8.
go back to reference Dehuri S, Cho S (2010) A hybrid genetic based functional link artificial neural network with a statistical comparison of classifiers over multiple datasets. Neural Comput Appl 19(2):317–328CrossRef Dehuri S, Cho S (2010) A hybrid genetic based functional link artificial neural network with a statistical comparison of classifiers over multiple datasets. Neural Comput Appl 19(2):317–328CrossRef
9.
go back to reference Zhang L, Suganthan PN (2016) A comprehensive evaluation of random vector functional link networks. Inf Sci 367:1094–1105CrossRef Zhang L, Suganthan PN (2016) A comprehensive evaluation of random vector functional link networks. Inf Sci 367:1094–1105CrossRef
10.
go back to reference Guo P, Lyu MR (2004) A pseudoinverse learning algorithm for feedforward neural networks with stacked generalization applications to software reliability growth data. Neurocomputing 56:101–121CrossRef Guo P, Lyu MR (2004) A pseudoinverse learning algorithm for feedforward neural networks with stacked generalization applications to software reliability growth data. Neurocomputing 56:101–121CrossRef
13.
go back to reference Pratama M, Angelov P, Lughofer E, Er MJ (2018) Parsimonious random vector functional link network for data streams. Inf Sci 430:519–537CrossRef Pratama M, Angelov P, Lughofer E, Er MJ (2018) Parsimonious random vector functional link network for data streams. Inf Sci 430:519–537CrossRef
14.
go back to reference Colace F, Loia V, Pedrycz W, Tomasiello S (2020) On a granular functional link network for classification. Neurocomputing 398:108–116CrossRef Colace F, Loia V, Pedrycz W, Tomasiello S (2020) On a granular functional link network for classification. Neurocomputing 398:108–116CrossRef
15.
go back to reference Zhang P, Yang Z (2020) A new learning paradigm for random vector functional-link network: RVFL+. Neural Netw 122:94–105CrossRef Zhang P, Yang Z (2020) A new learning paradigm for random vector functional-link network: RVFL+. Neural Netw 122:94–105CrossRef
16.
go back to reference Scardapane S, Comminiello D, Scarpiniti M, Uncini A (2016) A semi-supervised random vector functional-link network based on the transductive framework. Inf Sci 364–365:156–166CrossRefMATH Scardapane S, Comminiello D, Scarpiniti M, Uncini A (2016) A semi-supervised random vector functional-link network based on the transductive framework. Inf Sci 364–365:156–166CrossRefMATH
17.
go back to reference Guan S, Cui Z (2020) Modeling uncertain processes with interval random vector functional-link networks. J Process Control 93:43–52CrossRef Guan S, Cui Z (2020) Modeling uncertain processes with interval random vector functional-link networks. J Process Control 93:43–52CrossRef
18.
go back to reference Shi Q, Katuwal R, Suganthan PN, Tanveer M (2021) Random vector functional link neural network based ensemble deep learning. Pattern Recognit 117:107978CrossRef Shi Q, Katuwal R, Suganthan PN, Tanveer M (2021) Random vector functional link neural network based ensemble deep learning. Pattern Recognit 117:107978CrossRef
19.
go back to reference Katuwal R, Suganthan P (2019) Stacked autoencoder based deep random vector functional link neural network for classification. Appl Soft Comput 85:105854CrossRef Katuwal R, Suganthan P (2019) Stacked autoencoder based deep random vector functional link neural network for classification. Appl Soft Comput 85:105854CrossRef
20.
go back to reference Xie J, Liu S, Dai H, Rong Y (2020) Distributed semi-supervised learning algorithms for random vector functional-link networks with distributed data splitting across samples and features. Knowl Based Syst 195:105577CrossRef Xie J, Liu S, Dai H, Rong Y (2020) Distributed semi-supervised learning algorithms for random vector functional-link networks with distributed data splitting across samples and features. Knowl Based Syst 195:105577CrossRef
21.
go back to reference Vukovic N, Petrovic M, Miljkovic Z (2017) A comprehensive experimental evaluation of orthogonal polynomial expanded random vector functional link neural networks for regression. Appl Soft Comput 70:1083–1096CrossRef Vukovic N, Petrovic M, Miljkovic Z (2017) A comprehensive experimental evaluation of orthogonal polynomial expanded random vector functional link neural networks for regression. Appl Soft Comput 70:1083–1096CrossRef
22.
go back to reference Nayak DR, Dash R, Majhi B, Pachori RB, Zhang Y (2020) A deep stacked random vector functional link network autoencoder for diagnosis of brain abnormalities and breast cancer. Biomed Signal Process Control 58:101860CrossRef Nayak DR, Dash R, Majhi B, Pachori RB, Zhang Y (2020) A deep stacked random vector functional link network autoencoder for diagnosis of brain abnormalities and breast cancer. Biomed Signal Process Control 58:101860CrossRef
23.
go back to reference Tyukin I, Prokhorov DV (2009) In: Proceedings of the IEEE international conference on control applications, CCA 2009 and of the international symposium on intelligent control, ISIC 2009, Saint Petersburg, Russia, July 8-10, 2009, pp. 1391–1396. https://doi.org/10.1109/CCA.2009.5281061 Tyukin I, Prokhorov DV (2009) In: Proceedings of the IEEE international conference on control applications, CCA 2009 and of the international symposium on intelligent control, ISIC 2009, Saint Petersburg, Russia, July 8-10, 2009, pp. 1391–1396. https://​doi.​org/​10.​1109/​CCA.​2009.​5281061
24.
go back to reference Li M, Wang D (2017) Insights into randomized algorithms for neural networks: practical issues and common pitfalls. Inf Sci 382–383:170–178CrossRefMATH Li M, Wang D (2017) Insights into randomized algorithms for neural networks: practical issues and common pitfalls. Inf Sci 382–383:170–178CrossRefMATH
25.
go back to reference Shobana J, Murali M (2021) An efficient sentiment analysis methodology based on long short-term memory networks. Complex Intell Syst 7:2485–2501CrossRef Shobana J, Murali M (2021) An efficient sentiment analysis methodology based on long short-term memory networks. Complex Intell Syst 7:2485–2501CrossRef
26.
go back to reference Zhang Y, Wu J, Cai Z, Du B, Yu PS (2019) An unsupervised parameter learning model for RVFL neural network. Neural Netw 112:85–97CrossRefMATH Zhang Y, Wu J, Cai Z, Du B, Yu PS (2019) An unsupervised parameter learning model for RVFL neural network. Neural Netw 112:85–97CrossRefMATH
27.
go back to reference Paul AN, Yan P, Yang Y, Zhang H, Du S, Wu QMJ (2021) Non-iterative online sequential learning strategy for autoencoder and classifier. Neural Comput Appl 33(23):16345–16361CrossRef Paul AN, Yan P, Yang Y, Zhang H, Du S, Wu QMJ (2021) Non-iterative online sequential learning strategy for autoencoder and classifier. Neural Comput Appl 33(23):16345–16361CrossRef
28.
go back to reference Giryes R, Sapiro G, Bronstein AM (2016) Deep neural networks with random gaussian weights: A universal classification strategy? IEEE Trans Signal Process 64(13):3444–3457MathSciNetCrossRefMATH Giryes R, Sapiro G, Bronstein AM (2016) Deep neural networks with random gaussian weights: A universal classification strategy? IEEE Trans Signal Process 64(13):3444–3457MathSciNetCrossRefMATH
29.
go back to reference Guo P, Zhao D, Han M, Feng S (2019) In: Recent advances in big data and deep learning, proceedings of the INNS big data and deep learning conference INNSBDDL 2019, held at Sestri Levante, Genova, Italy 16-18 April 2019. Springer, pp. 158–168 Guo P, Zhao D, Han M, Feng S (2019) In: Recent advances in big data and deep learning, proceedings of the INNS big data and deep learning conference INNSBDDL 2019, held at Sestri Levante, Genova, Italy 16-18 April 2019. Springer, pp. 158–168
30.
go back to reference Wang K, Guo P (2021) A robust automated machine learning system with pseudoinverse learning. Cogn Comput 13(3):724–735CrossRef Wang K, Guo P (2021) A robust automated machine learning system with pseudoinverse learning. Cogn Comput 13(3):724–735CrossRef
31.
go back to reference Yin Q, Xu B, Zhou K, Guo P (2021) Bayesian pseudoinverse learners: from uncertainty to deterministic learning. IEEE Trans Cybern PP(99):1–12 Yin Q, Xu B, Zhou K, Guo P (2021) Bayesian pseudoinverse learners: from uncertainty to deterministic learning. IEEE Trans Cybern PP(99):1–12
32.
go back to reference Lee H, Kim N, Lee J (2017) Deep neural network self-training based on unsupervised learning and dropout. Int J Fuzzy Logic Intell Syst 17(1):1–9CrossRef Lee H, Kim N, Lee J (2017) Deep neural network self-training based on unsupervised learning and dropout. Int J Fuzzy Logic Intell Syst 17(1):1–9CrossRef
33.
go back to reference Guo P (2018) Building deep and broad learning systems based on pseudoinverse learning autoencoders. Special session presentation in CPCC 2018 (2018). In: The 29th Chinese process control conference (CPCC 2018). Shenyang Guo P (2018) Building deep and broad learning systems based on pseudoinverse learning autoencoders. Special session presentation in CPCC 2018 (2018). In: The 29th Chinese process control conference (CPCC 2018). Shenyang
35.
go back to reference Klambauer G, Unterthiner T, Mayr A, Hochreiter S (2017) In: Advances in neural information processing systems 30: annual conference on neural information processing systems 2017, 4-9 December 2017. Long Beach, CA, USA, pp. 971–980 Klambauer G, Unterthiner T, Mayr A, Hochreiter S (2017) In: Advances in neural information processing systems 30: annual conference on neural information processing systems 2017, 4-9 December 2017. Long Beach, CA, USA, pp. 971–980
36.
go back to reference He K, Zhang X, Ren S, Sun J (2015) In: 2015 IEEE international conference on computer vision, ICCV 2015. IEEE Computer Society, Santiago, pp. 1026–1034 He K, Zhang X, Ren S, Sun J (2015) In: 2015 IEEE international conference on computer vision, ICCV 2015. IEEE Computer Society, Santiago, pp. 1026–1034
37.
go back to reference Salimans T, Kingma DP (2016) In: Lee DD, Sugiyama M, von Luxburg U, Guyon I, Garnett R (eds) Advances in neural information processing systems 29: annual conference on neural information processing systems 2016, December 5-10, 2016. Barcelona, pp. 901 Salimans T, Kingma DP (2016) In: Lee DD, Sugiyama M, von Luxburg U, Guyon I, Garnett R (eds) Advances in neural information processing systems 29: annual conference on neural information processing systems 2016, December 5-10, 2016. Barcelona, pp. 901
38.
go back to reference Srivastava RK, Greff K, Schmidhuber J (2015) In: Cortes C, Lawrence ND, Lee DD, Sugiyama M, Garnett R (eds) Advances in neural information processing systems 28: annual conference on neural information processing systems 2015, December 7-12, 2015. Montreal, Quebec, pp. 2377–2385 Srivastava RK, Greff K, Schmidhuber J (2015) In: Cortes C, Lawrence ND, Lee DD, Sugiyama M, Garnett R (eds) Advances in neural information processing systems 28: annual conference on neural information processing systems 2015, December 7-12, 2015. Montreal, Quebec, pp. 2377–2385
39.
go back to reference Ba LJ, Kiros JR, Hinton GE (2016) Layer normalization. arXiv abs/1607.06450. 1607.06450 Ba LJ, Kiros JR, Hinton GE (2016) Layer normalization. arXiv abs/1607.06450. 1607.06450
40.
go back to reference Ioffe S, Szegedy C (2015) In: Bach FR, Blei DM (eds) Proceedings of the 32nd international conference on machine learning, ICML 2015, Lille, France, 6-11 July 2015, JMLR workshop and conference proceedings, vol. 37. pp. 448–456 Ioffe S, Szegedy C (2015) In: Bach FR, Blei DM (eds) Proceedings of the 32nd international conference on machine learning, ICML 2015, Lille, France, 6-11 July 2015, JMLR workshop and conference proceedings, vol. 37. pp. 448–456
41.
go back to reference He K, Zhang X, Ren S, Sun J (2016) In: 2016 IEEE conference on computer vision and pattern recognition, CVPR 2016. IEEE Computer Society, Las Vegas, pp. 770–778 He K, Zhang X, Ren S, Sun J (2016) In: 2016 IEEE conference on computer vision and pattern recognition, CVPR 2016. IEEE Computer Society, Las Vegas, pp. 770–778
Metadata
Title
An improved parameter learning methodology for RVFL based on pseudoinverse learners
Authors
Xiaoxuan Sun
Xiaodan Deng
Qian Yin
Ping Guo
Publication date
04-10-2022
Publisher
Springer London
Published in
Neural Computing and Applications / Issue 2/2023
Print ISSN: 0941-0643
Electronic ISSN: 1433-3058
DOI
https://doi.org/10.1007/s00521-022-07824-y

Other articles of this Issue 2/2023

Neural Computing and Applications 2/2023 Go to the issue

S.I. : 2019 India Intl. Congress on Computational Intelligence

A flexible framework for anomaly Detection via dimensionality reduction

Premium Partner