Skip to main content

2017 | OriginalPaper | Buchkapitel

Decouple Adversarial Capacities with Dual-Reservoir Network

verfasst von : Qianli Ma, Lifeng Shen, Wanqing Zhuang, Jieyu Chen

Erschienen in: Neural Information Processing

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Reservoir computing such as Echo State Network (ESN) and Liquid State Machine (LSM) has been successfully applied in dynamical system modeling. However, there is an antagonistic trade-off between the non-linear mapping capacity and the short-term memory capacity in single-reservoir networks, especially when the input signals contain high non-linearity and short-term dependencies. To address this problem, we propose a novel reservoir computing model called Dual-Reservoir Network (DRN), which connects two reservoirs with an unsupervised encoder such as PCA. Specifically, we allow these two adversarial capacities to be decoupled and enhanced in the dual reservoirs respectively. In our experiments, we first verify DRN’s feasibility on an extended polynomial system, which allows us to control the nonlinearity and short-term dependencies of data. In addition, we demonstrate the effectiveness of DRN on the synthesis and real-world time series predictions.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Bingham, E., Mannila, H.: Random projection in dimensionality reduction: applications to image and text data. In: Proceedings of the Seventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 245–250. ACM (2001) Bingham, E., Mannila, H.: Random projection in dimensionality reduction: applications to image and text data. In: Proceedings of the Seventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 245–250. ACM (2001)
2.
Zurück zum Zitat Butcher, J., Verstraeten, D., Schrauwen, B., Day, C., Haycock, P.: Reservoir computing and extreme learning machines for non-linear time-series data analysis. Neural Netw. 38, 76–89 (2013)CrossRef Butcher, J., Verstraeten, D., Schrauwen, B., Day, C., Haycock, P.: Reservoir computing and extreme learning machines for non-linear time-series data analysis. Neural Netw. 38, 76–89 (2013)CrossRef
3.
Zurück zum Zitat Butcher, J., Verstraeten, D., Schrauwen, B., Day, C., Haycock, P.: Extending reservoir computing with random static projections: a hybrid between extreme learning and RC. In: 18th European Symposium on Artificial Neural Networks (ESANN 2010), pp. 303–308. D-Side (2010) Butcher, J., Verstraeten, D., Schrauwen, B., Day, C., Haycock, P.: Extending reservoir computing with random static projections: a hybrid between extreme learning and RC. In: 18th European Symposium on Artificial Neural Networks (ESANN 2010), pp. 303–308. D-Side (2010)
4.
Zurück zum Zitat Gallicchio, C., Micheli, A.: Architectural and markovian factors of echo state networks. Neural Netw. 24(5), 440–456 (2011)CrossRef Gallicchio, C., Micheli, A.: Architectural and markovian factors of echo state networks. Neural Netw. 24(5), 440–456 (2011)CrossRef
5.
Zurück zum Zitat Huang, G.B., Zhu, Q.Y., Siew, C.K.: Extreme learning machine: theory and applications. Neurocomputing 70(13), 489–501 (2006). Neural Networks Selected Papers from the 7th Brazilian Symposium on Neural Networks (SBRN 2004)CrossRef Huang, G.B., Zhu, Q.Y., Siew, C.K.: Extreme learning machine: theory and applications. Neurocomputing 70(13), 489–501 (2006). Neural Networks Selected Papers from the 7th Brazilian Symposium on Neural Networks (SBRN 2004)CrossRef
6.
Zurück zum Zitat Jaeger, H.: The echo state approach to analysing and training recurrent neural networks-with an erratum note. German National Research Center for Information Technology GMD Technical report, Bonn, Germany, vol. 148(34), p. 13 (2001) Jaeger, H.: The echo state approach to analysing and training recurrent neural networks-with an erratum note. German National Research Center for Information Technology GMD Technical report, Bonn, Germany, vol. 148(34), p. 13 (2001)
7.
Zurück zum Zitat Jaeger, H., Haas, H.: Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication. Science 304(5667), 78–80 (2004)CrossRef Jaeger, H., Haas, H.: Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication. Science 304(5667), 78–80 (2004)CrossRef
8.
Zurück zum Zitat Jaeger, H., Lukoeviius, M., Popovici, D., Siewert, U.: Optimization and applications of echo state networks with leaky-integrator neurons. Neural Netw. 20(3), 335–352 (2007). Echo State Networks and Liquid State MachinesCrossRefMATH Jaeger, H., Lukoeviius, M., Popovici, D., Siewert, U.: Optimization and applications of echo state networks with leaky-integrator neurons. Neural Netw. 20(3), 335–352 (2007). Echo State Networks and Liquid State MachinesCrossRefMATH
9.
Zurück zum Zitat LukošEvičIus, M., Jaeger, H.: Reservoir computing approaches to recurrent neural network training. Comput. Sci. Rev. 3(3), 127–149 (2009)CrossRefMATH LukošEvičIus, M., Jaeger, H.: Reservoir computing approaches to recurrent neural network training. Comput. Sci. Rev. 3(3), 127–149 (2009)CrossRefMATH
10.
Zurück zum Zitat Tang, J., Deng, C., Huang, G.B.: Extreme learning machine for multilayer perceptron. IEEE Trans. Neural Netw. Learn. Syst. 27(4), 809–821 (2016)MathSciNetCrossRef Tang, J., Deng, C., Huang, G.B.: Extreme learning machine for multilayer perceptron. IEEE Trans. Neural Netw. Learn. Syst. 27(4), 809–821 (2016)MathSciNetCrossRef
12.
Zurück zum Zitat Verstraeten, D., Dambre, J., Dutoit, X., Schrauwen, B.: Memory versus non-linearity in reservoirs. In: The 2010 International Joint Conference on Neural Networks (IJCNN), pp. 1–8. IEEE (2010) Verstraeten, D., Dambre, J., Dutoit, X., Schrauwen, B.: Memory versus non-linearity in reservoirs. In: The 2010 International Joint Conference on Neural Networks (IJCNN), pp. 1–8. IEEE (2010)
13.
Zurück zum Zitat Wold, S., Esbensen, K., Geladi, P.: Principal component analysis. Chemometr. Intell. Lab. Syst. 2(1), 37–52 (1987). Proceedings of the Multivariate Statistical Workshop for Geologists and GeochemistsCrossRef Wold, S., Esbensen, K., Geladi, P.: Principal component analysis. Chemometr. Intell. Lab. Syst. 2(1), 37–52 (1987). Proceedings of the Multivariate Statistical Workshop for Geologists and GeochemistsCrossRef
Metadaten
Titel
Decouple Adversarial Capacities with Dual-Reservoir Network
verfasst von
Qianli Ma
Lifeng Shen
Wanqing Zhuang
Jieyu Chen
Copyright-Jahr
2017
DOI
https://doi.org/10.1007/978-3-319-70139-4_48