Skip to main content
Erschienen in: Neural Computing and Applications 2/2023

04.10.2022 | Original Article

An improved parameter learning methodology for RVFL based on pseudoinverse learners

verfasst von: Xiaoxuan Sun, Xiaodan Deng, Qian Yin, Ping Guo

Erschienen in: Neural Computing and Applications | Ausgabe 2/2023

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

As a compact and effective learning model, the random vector functional link neural network (RVFL) has been confirmed with universal approximation capabilities. It has gained considerable attention in various fields. However, the randomly generated parameters in RVFL often lead to the loss of valid information and data redundancy, which severely degrades the model performance in practice. This paper first proposes an efficient network parameters learning approach for the original RVFL with pseudoinverse learner (RVFL-PL). Instead of taking the random feature mapping directly, RVFL-PL adopts a non-iterative manner to obtain influential enhancement nodes implanted with valuable information from input data, which realizes to improve the quality of the enhancement nodes and ease the problem caused by the randomly assigned parameters in the standard RVFL. Since the network parameters are optimized analytically, this improved variant can maintain the efficiency of the standard RVFL. Further, the RVFL-PL is extended to a multilayered structure (mRVFL-PL) to obtain high-level representations from the input data. The results of comprehensive experiments on some benchmarks indicate the performance improvement of the proposed method compared to other corresponding methods.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Yue K, Xu F, Yu J (2019) Shallow and wide fractional max-pooling network for image classification. Neural Comput Appl 31(2):409–419CrossRef Yue K, Xu F, Yu J (2019) Shallow and wide fractional max-pooling network for image classification. Neural Comput Appl 31(2):409–419CrossRef
2.
Zurück zum Zitat Jia Y, Chen X, Yu J, Wang L, Wang Y (2021) Speaker recognition based on characteristic spectrograms and an improved self-organizing feature map neural network. Complex Intell Syst 7:1749–1757CrossRef Jia Y, Chen X, Yu J, Wang L, Wang Y (2021) Speaker recognition based on characteristic spectrograms and an improved self-organizing feature map neural network. Complex Intell Syst 7:1749–1757CrossRef
5.
Zurück zum Zitat Cao W, Wang X, Ming Z, Gao J (2018) A review on neural networks with random weights. Neurocomputing 275:278–287CrossRef Cao W, Wang X, Ming Z, Gao J (2018) A review on neural networks with random weights. Neurocomputing 275:278–287CrossRef
6.
Zurück zum Zitat Zhang L, Suganthan PN (2016) A survey of randomized algorithms for training neural networks. Inf Sci 364:146–155CrossRefMATH Zhang L, Suganthan PN (2016) A survey of randomized algorithms for training neural networks. Inf Sci 364:146–155CrossRefMATH
7.
Zurück zum Zitat Pao Y, Park GH, Sobajic DJ (1994) Learning and generalization characteristics of the random vector functional-link net. Neurocomputing 6(2):163–180CrossRef Pao Y, Park GH, Sobajic DJ (1994) Learning and generalization characteristics of the random vector functional-link net. Neurocomputing 6(2):163–180CrossRef
8.
Zurück zum Zitat Dehuri S, Cho S (2010) A hybrid genetic based functional link artificial neural network with a statistical comparison of classifiers over multiple datasets. Neural Comput Appl 19(2):317–328CrossRef Dehuri S, Cho S (2010) A hybrid genetic based functional link artificial neural network with a statistical comparison of classifiers over multiple datasets. Neural Comput Appl 19(2):317–328CrossRef
9.
Zurück zum Zitat Zhang L, Suganthan PN (2016) A comprehensive evaluation of random vector functional link networks. Inf Sci 367:1094–1105CrossRef Zhang L, Suganthan PN (2016) A comprehensive evaluation of random vector functional link networks. Inf Sci 367:1094–1105CrossRef
10.
Zurück zum Zitat Guo P, Lyu MR (2004) A pseudoinverse learning algorithm for feedforward neural networks with stacked generalization applications to software reliability growth data. Neurocomputing 56:101–121CrossRef Guo P, Lyu MR (2004) A pseudoinverse learning algorithm for feedforward neural networks with stacked generalization applications to software reliability growth data. Neurocomputing 56:101–121CrossRef
13.
Zurück zum Zitat Pratama M, Angelov P, Lughofer E, Er MJ (2018) Parsimonious random vector functional link network for data streams. Inf Sci 430:519–537CrossRef Pratama M, Angelov P, Lughofer E, Er MJ (2018) Parsimonious random vector functional link network for data streams. Inf Sci 430:519–537CrossRef
14.
Zurück zum Zitat Colace F, Loia V, Pedrycz W, Tomasiello S (2020) On a granular functional link network for classification. Neurocomputing 398:108–116CrossRef Colace F, Loia V, Pedrycz W, Tomasiello S (2020) On a granular functional link network for classification. Neurocomputing 398:108–116CrossRef
15.
Zurück zum Zitat Zhang P, Yang Z (2020) A new learning paradigm for random vector functional-link network: RVFL+. Neural Netw 122:94–105CrossRef Zhang P, Yang Z (2020) A new learning paradigm for random vector functional-link network: RVFL+. Neural Netw 122:94–105CrossRef
16.
Zurück zum Zitat Scardapane S, Comminiello D, Scarpiniti M, Uncini A (2016) A semi-supervised random vector functional-link network based on the transductive framework. Inf Sci 364–365:156–166CrossRefMATH Scardapane S, Comminiello D, Scarpiniti M, Uncini A (2016) A semi-supervised random vector functional-link network based on the transductive framework. Inf Sci 364–365:156–166CrossRefMATH
17.
Zurück zum Zitat Guan S, Cui Z (2020) Modeling uncertain processes with interval random vector functional-link networks. J Process Control 93:43–52CrossRef Guan S, Cui Z (2020) Modeling uncertain processes with interval random vector functional-link networks. J Process Control 93:43–52CrossRef
18.
Zurück zum Zitat Shi Q, Katuwal R, Suganthan PN, Tanveer M (2021) Random vector functional link neural network based ensemble deep learning. Pattern Recognit 117:107978CrossRef Shi Q, Katuwal R, Suganthan PN, Tanveer M (2021) Random vector functional link neural network based ensemble deep learning. Pattern Recognit 117:107978CrossRef
19.
Zurück zum Zitat Katuwal R, Suganthan P (2019) Stacked autoencoder based deep random vector functional link neural network for classification. Appl Soft Comput 85:105854CrossRef Katuwal R, Suganthan P (2019) Stacked autoencoder based deep random vector functional link neural network for classification. Appl Soft Comput 85:105854CrossRef
20.
Zurück zum Zitat Xie J, Liu S, Dai H, Rong Y (2020) Distributed semi-supervised learning algorithms for random vector functional-link networks with distributed data splitting across samples and features. Knowl Based Syst 195:105577CrossRef Xie J, Liu S, Dai H, Rong Y (2020) Distributed semi-supervised learning algorithms for random vector functional-link networks with distributed data splitting across samples and features. Knowl Based Syst 195:105577CrossRef
21.
Zurück zum Zitat Vukovic N, Petrovic M, Miljkovic Z (2017) A comprehensive experimental evaluation of orthogonal polynomial expanded random vector functional link neural networks for regression. Appl Soft Comput 70:1083–1096CrossRef Vukovic N, Petrovic M, Miljkovic Z (2017) A comprehensive experimental evaluation of orthogonal polynomial expanded random vector functional link neural networks for regression. Appl Soft Comput 70:1083–1096CrossRef
22.
Zurück zum Zitat Nayak DR, Dash R, Majhi B, Pachori RB, Zhang Y (2020) A deep stacked random vector functional link network autoencoder for diagnosis of brain abnormalities and breast cancer. Biomed Signal Process Control 58:101860CrossRef Nayak DR, Dash R, Majhi B, Pachori RB, Zhang Y (2020) A deep stacked random vector functional link network autoencoder for diagnosis of brain abnormalities and breast cancer. Biomed Signal Process Control 58:101860CrossRef
23.
Zurück zum Zitat Tyukin I, Prokhorov DV (2009) In: Proceedings of the IEEE international conference on control applications, CCA 2009 and of the international symposium on intelligent control, ISIC 2009, Saint Petersburg, Russia, July 8-10, 2009, pp. 1391–1396. https://doi.org/10.1109/CCA.2009.5281061 Tyukin I, Prokhorov DV (2009) In: Proceedings of the IEEE international conference on control applications, CCA 2009 and of the international symposium on intelligent control, ISIC 2009, Saint Petersburg, Russia, July 8-10, 2009, pp. 1391–1396. https://​doi.​org/​10.​1109/​CCA.​2009.​5281061
24.
Zurück zum Zitat Li M, Wang D (2017) Insights into randomized algorithms for neural networks: practical issues and common pitfalls. Inf Sci 382–383:170–178CrossRefMATH Li M, Wang D (2017) Insights into randomized algorithms for neural networks: practical issues and common pitfalls. Inf Sci 382–383:170–178CrossRefMATH
25.
Zurück zum Zitat Shobana J, Murali M (2021) An efficient sentiment analysis methodology based on long short-term memory networks. Complex Intell Syst 7:2485–2501CrossRef Shobana J, Murali M (2021) An efficient sentiment analysis methodology based on long short-term memory networks. Complex Intell Syst 7:2485–2501CrossRef
26.
Zurück zum Zitat Zhang Y, Wu J, Cai Z, Du B, Yu PS (2019) An unsupervised parameter learning model for RVFL neural network. Neural Netw 112:85–97CrossRefMATH Zhang Y, Wu J, Cai Z, Du B, Yu PS (2019) An unsupervised parameter learning model for RVFL neural network. Neural Netw 112:85–97CrossRefMATH
27.
Zurück zum Zitat Paul AN, Yan P, Yang Y, Zhang H, Du S, Wu QMJ (2021) Non-iterative online sequential learning strategy for autoencoder and classifier. Neural Comput Appl 33(23):16345–16361CrossRef Paul AN, Yan P, Yang Y, Zhang H, Du S, Wu QMJ (2021) Non-iterative online sequential learning strategy for autoencoder and classifier. Neural Comput Appl 33(23):16345–16361CrossRef
28.
Zurück zum Zitat Giryes R, Sapiro G, Bronstein AM (2016) Deep neural networks with random gaussian weights: A universal classification strategy? IEEE Trans Signal Process 64(13):3444–3457MathSciNetCrossRefMATH Giryes R, Sapiro G, Bronstein AM (2016) Deep neural networks with random gaussian weights: A universal classification strategy? IEEE Trans Signal Process 64(13):3444–3457MathSciNetCrossRefMATH
29.
Zurück zum Zitat Guo P, Zhao D, Han M, Feng S (2019) In: Recent advances in big data and deep learning, proceedings of the INNS big data and deep learning conference INNSBDDL 2019, held at Sestri Levante, Genova, Italy 16-18 April 2019. Springer, pp. 158–168 Guo P, Zhao D, Han M, Feng S (2019) In: Recent advances in big data and deep learning, proceedings of the INNS big data and deep learning conference INNSBDDL 2019, held at Sestri Levante, Genova, Italy 16-18 April 2019. Springer, pp. 158–168
30.
Zurück zum Zitat Wang K, Guo P (2021) A robust automated machine learning system with pseudoinverse learning. Cogn Comput 13(3):724–735CrossRef Wang K, Guo P (2021) A robust automated machine learning system with pseudoinverse learning. Cogn Comput 13(3):724–735CrossRef
31.
Zurück zum Zitat Yin Q, Xu B, Zhou K, Guo P (2021) Bayesian pseudoinverse learners: from uncertainty to deterministic learning. IEEE Trans Cybern PP(99):1–12 Yin Q, Xu B, Zhou K, Guo P (2021) Bayesian pseudoinverse learners: from uncertainty to deterministic learning. IEEE Trans Cybern PP(99):1–12
32.
Zurück zum Zitat Lee H, Kim N, Lee J (2017) Deep neural network self-training based on unsupervised learning and dropout. Int J Fuzzy Logic Intell Syst 17(1):1–9CrossRef Lee H, Kim N, Lee J (2017) Deep neural network self-training based on unsupervised learning and dropout. Int J Fuzzy Logic Intell Syst 17(1):1–9CrossRef
33.
Zurück zum Zitat Guo P (2018) Building deep and broad learning systems based on pseudoinverse learning autoencoders. Special session presentation in CPCC 2018 (2018). In: The 29th Chinese process control conference (CPCC 2018). Shenyang Guo P (2018) Building deep and broad learning systems based on pseudoinverse learning autoencoders. Special session presentation in CPCC 2018 (2018). In: The 29th Chinese process control conference (CPCC 2018). Shenyang
35.
Zurück zum Zitat Klambauer G, Unterthiner T, Mayr A, Hochreiter S (2017) In: Advances in neural information processing systems 30: annual conference on neural information processing systems 2017, 4-9 December 2017. Long Beach, CA, USA, pp. 971–980 Klambauer G, Unterthiner T, Mayr A, Hochreiter S (2017) In: Advances in neural information processing systems 30: annual conference on neural information processing systems 2017, 4-9 December 2017. Long Beach, CA, USA, pp. 971–980
36.
Zurück zum Zitat He K, Zhang X, Ren S, Sun J (2015) In: 2015 IEEE international conference on computer vision, ICCV 2015. IEEE Computer Society, Santiago, pp. 1026–1034 He K, Zhang X, Ren S, Sun J (2015) In: 2015 IEEE international conference on computer vision, ICCV 2015. IEEE Computer Society, Santiago, pp. 1026–1034
37.
Zurück zum Zitat Salimans T, Kingma DP (2016) In: Lee DD, Sugiyama M, von Luxburg U, Guyon I, Garnett R (eds) Advances in neural information processing systems 29: annual conference on neural information processing systems 2016, December 5-10, 2016. Barcelona, pp. 901 Salimans T, Kingma DP (2016) In: Lee DD, Sugiyama M, von Luxburg U, Guyon I, Garnett R (eds) Advances in neural information processing systems 29: annual conference on neural information processing systems 2016, December 5-10, 2016. Barcelona, pp. 901
38.
Zurück zum Zitat Srivastava RK, Greff K, Schmidhuber J (2015) In: Cortes C, Lawrence ND, Lee DD, Sugiyama M, Garnett R (eds) Advances in neural information processing systems 28: annual conference on neural information processing systems 2015, December 7-12, 2015. Montreal, Quebec, pp. 2377–2385 Srivastava RK, Greff K, Schmidhuber J (2015) In: Cortes C, Lawrence ND, Lee DD, Sugiyama M, Garnett R (eds) Advances in neural information processing systems 28: annual conference on neural information processing systems 2015, December 7-12, 2015. Montreal, Quebec, pp. 2377–2385
39.
Zurück zum Zitat Ba LJ, Kiros JR, Hinton GE (2016) Layer normalization. arXiv abs/1607.06450. 1607.06450 Ba LJ, Kiros JR, Hinton GE (2016) Layer normalization. arXiv abs/1607.06450. 1607.06450
40.
Zurück zum Zitat Ioffe S, Szegedy C (2015) In: Bach FR, Blei DM (eds) Proceedings of the 32nd international conference on machine learning, ICML 2015, Lille, France, 6-11 July 2015, JMLR workshop and conference proceedings, vol. 37. pp. 448–456 Ioffe S, Szegedy C (2015) In: Bach FR, Blei DM (eds) Proceedings of the 32nd international conference on machine learning, ICML 2015, Lille, France, 6-11 July 2015, JMLR workshop and conference proceedings, vol. 37. pp. 448–456
41.
Zurück zum Zitat He K, Zhang X, Ren S, Sun J (2016) In: 2016 IEEE conference on computer vision and pattern recognition, CVPR 2016. IEEE Computer Society, Las Vegas, pp. 770–778 He K, Zhang X, Ren S, Sun J (2016) In: 2016 IEEE conference on computer vision and pattern recognition, CVPR 2016. IEEE Computer Society, Las Vegas, pp. 770–778
Metadaten
Titel
An improved parameter learning methodology for RVFL based on pseudoinverse learners
verfasst von
Xiaoxuan Sun
Xiaodan Deng
Qian Yin
Ping Guo
Publikationsdatum
04.10.2022
Verlag
Springer London
Erschienen in
Neural Computing and Applications / Ausgabe 2/2023
Print ISSN: 0941-0643
Elektronische ISSN: 1433-3058
DOI
https://doi.org/10.1007/s00521-022-07824-y

Weitere Artikel der Ausgabe 2/2023

Neural Computing and Applications 2/2023 Zur Ausgabe

Premium Partner