Skip to main content
Top
Published in: Cognitive Computation 5/2023

15-02-2021

Hardware-Optimized Reservoir Computing System for Edge Intelligence Applications

Authors: Alejandro Morán, Vincent Canals, Fabio Galan-Prado, Christian F. Frasser, Dhinakar Radhakrishnan, Saeid Safavi, Josep L. Rosselló

Published in: Cognitive Computation | Issue 5/2023

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Edge artificial intelligence or edge intelligence is an ever-growing research area due to the current popularization of the Internet of Things. Unfortunately, incorporation of artificial intelligence (AI) in smart devices operating at the edge is a challenging task due to the power-hungry characteristics of deep learning implementations, such as convolutional neural networks (CNNs). As a feasible alternative, reservoir computing (RC) has attracted a lot of attention in the field of machine learning due to its promising performance in a wide range of applications. In this work, we propose a simple hardware-optimized circuit design of RC systems presenting high energy-efficiency capacities that fulfill the low power requirements of edge intelligence applications. As a proof of concept, we used the proposed design for the implementation of a low-power audio event detection (AED) application in FPGA. The measurements and simulation results obtained show that the proposed approach may provide significant accuracy with the advantage of presenting ultra-low-power characteristics (the energy efficiency estimated is below the microjoule per inference). These results make the proposed system optimal for edge intelligence applications in which energy efficiency and accuracy are the key issues.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Footnotes
1
Top-N accuracy is computed by interpreting as correct those predictions for which the ground truth is one of the N most likely categories.
 
Literature
1.
go back to reference Chen D, Cong J, Gurumani S, Wm Hwu, Rupnow K, Zhang Z. Platform choices and design demands for IoT platforms: cost, power, and performance tradeoffs. IET Cyber-Physical Systems: Theory & Applications 2016;1(1):70–77.CrossRef Chen D, Cong J, Gurumani S, Wm Hwu, Rupnow K, Zhang Z. Platform choices and design demands for IoT platforms: cost, power, and performance tradeoffs. IET Cyber-Physical Systems: Theory & Applications 2016;1(1):70–77.CrossRef
2.
go back to reference O’Leary DE. Artificial intelligence and big data. IEEE Intelligent Systems 2013;28(2):96–99.CrossRef O’Leary DE. Artificial intelligence and big data. IEEE Intelligent Systems 2013;28(2):96–99.CrossRef
3.
go back to reference Silva DA, Stuchi JA, Violato RPV, Cuozzo LGD. Exploring convolutional neural networks for voice activity detection. Cognitive technologies. Springer; 2017. p. 37–47. Silva DA, Stuchi JA, Violato RPV, Cuozzo LGD. Exploring convolutional neural networks for voice activity detection. Cognitive technologies. Springer; 2017. p. 37–47.
4.
go back to reference Zhang Y, Suda N, Lai L, Chandra V. 2017. Hello edge: keyword spotting on microcontrollers. arXiv:171107128. Zhang Y, Suda N, Lai L, Chandra V. 2017. Hello edge: keyword spotting on microcontrollers. arXiv:171107128.
5.
go back to reference Liu B, Wang Z, Zhu W, Sun Y, Shen Z, Huang L, Li Y, Gong Y, Ge W. An ultra-low power always-on keyword spotting accelerator using quantized convolutional neural network and voltage-domain analog switching network-based approximate computing. IEEE Access 2019;7:186456–186469.CrossRef Liu B, Wang Z, Zhu W, Sun Y, Shen Z, Huang L, Li Y, Gong Y, Ge W. An ultra-low power always-on keyword spotting accelerator using quantized convolutional neural network and voltage-domain analog switching network-based approximate computing. IEEE Access 2019;7:186456–186469.CrossRef
6.
go back to reference Maass W, Natschläger T, Markram H. Real-time computing without stable states: a new framework for neural computation based on perturbations. Neural Computation 2002;14(11):2531–2560.CrossRefMATH Maass W, Natschläger T, Markram H. Real-time computing without stable states: a new framework for neural computation based on perturbations. Neural Computation 2002;14(11):2531–2560.CrossRefMATH
7.
go back to reference Jaeger H, Haas H. Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication. Science 2004;304(5667):78–80.CrossRef Jaeger H, Haas H. Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication. Science 2004;304(5667):78–80.CrossRef
8.
go back to reference Lukoševičius M, Jaeger H. Reservoir computing approaches to recurrent neural network training. Computer Science Review 2009;3(3):127–149.CrossRefMATH Lukoševičius M, Jaeger H. Reservoir computing approaches to recurrent neural network training. Computer Science Review 2009;3(3):127–149.CrossRefMATH
9.
go back to reference Maass W. Liquid state machines: motivation, theory, and applications. Computability in context: computation and logic in the real world. World Scientific; 2011. p. 275–296. Maass W. Liquid state machines: motivation, theory, and applications. Computability in context: computation and logic in the real world. World Scientific; 2011. p. 275–296.
10.
go back to reference Appeltant L, Soriano MC, Van der Sande G, Danckaert J, Massar S, Dambre J, Schrauwen B, Mirasso CR, Fischer I. Information processing using a single dynamical node as complex system. Nature Communications 2011;2(1):1–6.CrossRef Appeltant L, Soriano MC, Van der Sande G, Danckaert J, Massar S, Dambre J, Schrauwen B, Mirasso CR, Fischer I. Information processing using a single dynamical node as complex system. Nature Communications 2011;2(1):1–6.CrossRef
11.
go back to reference Scardapane S, Butcher J, Bianchi F, Malik Z. Advances in biologically inspired reservoir computing. Cognitive Computation 2017;9(3):295–296.CrossRef Scardapane S, Butcher J, Bianchi F, Malik Z. Advances in biologically inspired reservoir computing. Cognitive Computation 2017;9(3):295–296.CrossRef
12.
go back to reference Katumba A, Freiberger M, Bienstman P, Dambre J. A multiple-input strategy to efficient integrated photonic reservoir computing. Cognitive Computation 2017;9(3):307–314.CrossRef Katumba A, Freiberger M, Bienstman P, Dambre J. A multiple-input strategy to efficient integrated photonic reservoir computing. Cognitive Computation 2017;9(3):307–314.CrossRef
13.
go back to reference Gallicchio C, Micheli A. Echo state property of deep reservoir computing networks. Cognitive Computation 2017;9(3):337–350.CrossRef Gallicchio C, Micheli A. Echo state property of deep reservoir computing networks. Cognitive Computation 2017;9(3):337–350.CrossRef
14.
go back to reference Alomar M, Skibinsky-Gitlin ES, Frasser CF, Canals V, Isern E, Roca M, Rosselló JL. 2017. Efficient parallel implementation of reservoir computing systems. Neural Comput & Applic: 32 1–15. Alomar M, Skibinsky-Gitlin ES, Frasser CF, Canals V, Isern E, Roca M, Rosselló JL. 2017. Efficient parallel implementation of reservoir computing systems. Neural Comput & Applic: 32 1–15.
15.
go back to reference Rodan A, Tino P. Minimum complexity echo state network. IEEE Trans Neural Netw 2010; 22(1):131–144.CrossRef Rodan A, Tino P. Minimum complexity echo state network. IEEE Trans Neural Netw 2010; 22(1):131–144.CrossRef
16.
go back to reference Alomar ML, Soriano MC, Escalona-Morán M, Canals V, Fischer I, Mirasso CR, Rosselló JL. Digital implementation of a single dynamical node reservoir computer. IEEE Transactions on Circuits and Systems II: Express Briefs 2015;62(10):977–981. Alomar ML, Soriano MC, Escalona-Morán M, Canals V, Fischer I, Mirasso CR, Rosselló JL. Digital implementation of a single dynamical node reservoir computer. IEEE Transactions on Circuits and Systems II: Express Briefs 2015;62(10):977–981.
17.
go back to reference Penkovsky B, Larger L, Brunner D. Efficient design of hardware-enabled reservoir computing in fpgas. Journal of Applied Physics 2018;124(16):162101.CrossRef Penkovsky B, Larger L, Brunner D. Efficient design of hardware-enabled reservoir computing in fpgas. Journal of Applied Physics 2018;124(16):162101.CrossRef
18.
go back to reference Larger L, Soriano MC, Brunner D, Appeltant L, Gutiérrez JM, Pesquera L, Mirasso CR, Fischer I. Photonic information processing beyond turing: an optoelectronic implementation of reservoir computing. Optics Express 2012;20(3):3241–3249.CrossRef Larger L, Soriano MC, Brunner D, Appeltant L, Gutiérrez JM, Pesquera L, Mirasso CR, Fischer I. Photonic information processing beyond turing: an optoelectronic implementation of reservoir computing. Optics Express 2012;20(3):3241–3249.CrossRef
19.
go back to reference Skibinsky-Gitlin ES, Alomar ML, Isern E, Roca M, Canals V, Rossello JL. Reservoir computing hardware for time series forecasting. 2018 28th international symposium on power and timing modeling, optimization and simulation (PATMOS). IEEE; 2018. p. 133–139. Skibinsky-Gitlin ES, Alomar ML, Isern E, Roca M, Canals V, Rossello JL. Reservoir computing hardware for time series forecasting. 2018 28th international symposium on power and timing modeling, optimization and simulation (PATMOS). IEEE; 2018. p. 133–139.
20.
go back to reference Skibinsky-Gitlin ES, Alomar ML, Canals V, Frasser CF, Isern E, Galán-Prado F, Morán A, Roca M, Rosselló JL. Fpga-based echo-state networks. International conference on time series and forecasting. Springer; 2018. p. 135–146. Skibinsky-Gitlin ES, Alomar ML, Canals V, Frasser CF, Isern E, Galán-Prado F, Morán A, Roca M, Rosselló JL. Fpga-based echo-state networks. International conference on time series and forecasting. Springer; 2018. p. 135–146.
21.
go back to reference Salamon J, Jacoby C, Bello JP. A dataset and taxonomy for urban sound research. Proceedings of the 22nd ACM international conference on multimedia; 2014. p. 1041–1044. Salamon J, Jacoby C, Bello JP. A dataset and taxonomy for urban sound research. Proceedings of the 22nd ACM international conference on multimedia; 2014. p. 1041–1044.
22.
go back to reference Costoya AM, Frasser CF, Roca M, Rossello JL. 2019. Energy-efficient pattern recognition hardware with elementary cellular automata. IEEE Transactions on Computers. Costoya AM, Frasser CF, Roca M, Rossello JL. 2019. Energy-efficient pattern recognition hardware with elementary cellular automata. IEEE Transactions on Computers.
23.
go back to reference Jaeger H. 2001. The echo state approach to analysing and training recurrent neural networks. GMD Report. Jaeger H. 2001. The echo state approach to analysing and training recurrent neural networks. GMD Report.
24.
go back to reference Maass W, Natschläger T, Markram H. Real-time computing without stable states: a new framework for neural computation based on perturbations. Neural Comput 2002;14(11):2531– 2560.CrossRefMATH Maass W, Natschläger T, Markram H. Real-time computing without stable states: a new framework for neural computation based on perturbations. Neural Comput 2002;14(11):2531– 2560.CrossRefMATH
25.
go back to reference Matsubara T, Torikai H, Hishiki T. A generalized rotate-and-fire digital spiking neuron model and its on-fpga learning. IEEE Transactions on Circuits and Systems II: Express Briefs 2011;58(10):677–681. Matsubara T, Torikai H, Hishiki T. A generalized rotate-and-fire digital spiking neuron model and its on-fpga learning. IEEE Transactions on Circuits and Systems II: Express Briefs 2011;58(10):677–681.
26.
go back to reference Nouri M, Jalilian M, Hayati M, Abbott D. A digital neuromorphic realization of pair-based and triplet-based spike-timing-dependent synaptic plasticity. IEEE Transactions on Circuits and Systems II: Express Briefs 2018;65(6):804–808. Nouri M, Jalilian M, Hayati M, Abbott D. A digital neuromorphic realization of pair-based and triplet-based spike-timing-dependent synaptic plasticity. IEEE Transactions on Circuits and Systems II: Express Briefs 2018;65(6):804–808.
27.
28.
go back to reference Corporation A. 2008. Nios ii processor reference handbook. Corporation A. 2008. Nios ii processor reference handbook.
29.
go back to reference Xilinx I. 2006. Microblaze processor reference guide. reference manual 23. Xilinx I. 2006. Microblaze processor reference guide. reference manual 23.
30.
go back to reference Han W, Chan CF, Choy CS, Pun KP. An efficient mfcc extraction method in speech recognition. 2006 IEEE international symposium on circuits and systems. IEEE; 2006. p. 4–pp. Han W, Chan CF, Choy CS, Pun KP. An efficient mfcc extraction method in speech recognition. 2006 IEEE international symposium on circuits and systems. IEEE; 2006. p. 4–pp.
31.
go back to reference Meyer M, Cavigelli L, Thiele L. 2017. Efficient convolutional neural network for audio event detection. arXiv:170909888. Meyer M, Cavigelli L, Thiele L. 2017. Efficient convolutional neural network for audio event detection. arXiv:170909888.
32.
go back to reference Salamon J, Bello JP. Deep convolutional neural networks and data augmentation for environmental sound classification. IEEE Signal Processing Letters 2017;24(3):279–283.CrossRef Salamon J, Bello JP. Deep convolutional neural networks and data augmentation for environmental sound classification. IEEE Signal Processing Letters 2017;24(3):279–283.CrossRef
33.
go back to reference Kücüktopcu O, Masazade E, Ünsalan C, Varshney PK. A real-time bird sound recognition system using a low-cost microcontroller. Appl Acoust 2019;148:194–201.CrossRef Kücüktopcu O, Masazade E, Ünsalan C, Varshney PK. A real-time bird sound recognition system using a low-cost microcontroller. Appl Acoust 2019;148:194–201.CrossRef
34.
go back to reference Schaetti N, Salomon M, Couturier R. Echo state networks-based reservoir computing for mnist handwritten digits recognition. 2016 IEEE Intl conference on computational science and engineering (CSE) and IEEE intl conference on embedded and ubiquitous computing (EUC) and 15th intl symposium on distributed computing and applications for business engineering (DCABES). IEEE; 2016. p. 484–491. Schaetti N, Salomon M, Couturier R. Echo state networks-based reservoir computing for mnist handwritten digits recognition. 2016 IEEE Intl conference on computational science and engineering (CSE) and IEEE intl conference on embedded and ubiquitous computing (EUC) and 15th intl symposium on distributed computing and applications for business engineering (DCABES). IEEE; 2016. p. 484–491.
35.
go back to reference Saleh R, Wilton S, Mirabbasi S, Hu A, Greenstreet M, Lemieux G, Pande PP, Grecu C, Ivanov A. System-on-chip: reuse and integration. Proc IEEE 2006;94(6):1050–1069.CrossRef Saleh R, Wilton S, Mirabbasi S, Hu A, Greenstreet M, Lemieux G, Pande PP, Grecu C, Ivanov A. System-on-chip: reuse and integration. Proc IEEE 2006;94(6):1050–1069.CrossRef
36.
go back to reference Escalona-Morán MA, Soriano MC, Fischer I, Mirasso CR. Electrocardiogram classification using reservoir computing with logistic regression. IEEE Journal of Biomedical and health Informatics 2014;19 (3):892–898.CrossRef Escalona-Morán MA, Soriano MC, Fischer I, Mirasso CR. Electrocardiogram classification using reservoir computing with logistic regression. IEEE Journal of Biomedical and health Informatics 2014;19 (3):892–898.CrossRef
37.
go back to reference Patra JC, Pal RN. A functional link artificial neural network for adaptive channel equalization. Signal Process 1995;43(2):181–195.CrossRefMATH Patra JC, Pal RN. A functional link artificial neural network for adaptive channel equalization. Signal Process 1995;43(2):181–195.CrossRefMATH
38.
go back to reference Inoue T, Vinayavekhin P, Wang S, Wood D, Greco N, Tachibana R. 2018. Domestic activities classification based on cnn using shuffling and mixing data augmentation. DCASE 2018 Challenge. Inoue T, Vinayavekhin P, Wang S, Wood D, Greco N, Tachibana R. 2018. Domestic activities classification based on cnn using shuffling and mixing data augmentation. DCASE 2018 Challenge.
Metadata
Title
Hardware-Optimized Reservoir Computing System for Edge Intelligence Applications
Authors
Alejandro Morán
Vincent Canals
Fabio Galan-Prado
Christian F. Frasser
Dhinakar Radhakrishnan
Saeid Safavi
Josep L. Rosselló
Publication date
15-02-2021
Publisher
Springer US
Published in
Cognitive Computation / Issue 5/2023
Print ISSN: 1866-9956
Electronic ISSN: 1866-9964
DOI
https://doi.org/10.1007/s12559-020-09798-2

Other articles of this Issue 5/2023

Cognitive Computation 5/2023 Go to the issue

Premium Partner