Skip to main content
Erschienen in: Neural Processing Letters 1/2022

23.09.2021

Complex Valued Deep Neural Networks for Nonlinear System Modeling

verfasst von: Mario Lopez-Pacheco, Wen Yu

Erschienen in: Neural Processing Letters | Ausgabe 1/2022

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Deep learning models, such as convolutional neural networks (CNN), have been successfully applied in pattern recognition and system identification recent years. But for the cases of missing data and big noises, CNN does not work well for dynamic system modeling. In this paper, complex valued convolution neural network (CVCNN) is presented for modeling nonlinear systems with large uncertainties. Novel training methods are proposed for CVCNN. Comparisons with other classical neural networks are made to show the advantages of the proposed methods.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Noel JP, Kerschen G (2017) Nonlinear system identification in structural dynamics: 10 more years of progress. Mech Syst Signal Process 83:2–35CrossRef Noel JP, Kerschen G (2017) Nonlinear system identification in structural dynamics: 10 more years of progress. Mech Syst Signal Process 83:2–35CrossRef
2.
Zurück zum Zitat Lopez M, Morales J, Yu W (2020) Frequency domain CNN and disipate energy approach for damage detection in building structures. Soft Comput 24:15821–1584051CrossRef Lopez M, Morales J, Yu W (2020) Frequency domain CNN and disipate energy approach for damage detection in building structures. Soft Comput 24:15821–1584051CrossRef
3.
Zurück zum Zitat LeCun Y, Boser B, Denker JS, Henderson D, Howard RE, Hubbard W, Jackel LD (1989) Backpropagation applied to handwritten zip code recognition. Neural Comput 1(4):541–551CrossRef LeCun Y, Boser B, Denker JS, Henderson D, Howard RE, Hubbard W, Jackel LD (1989) Backpropagation applied to handwritten zip code recognition. Neural Comput 1(4):541–551CrossRef
4.
Zurück zum Zitat Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15(1):1929–1958MathSciNetMATH Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15(1):1929–1958MathSciNetMATH
5.
Zurück zum Zitat Bengio Y, Lamblin P, Popovici D, Larochelle H (2007) Greedy layer-wise training of deep networks. In: Proceedings of the advances in neural information processing systems (NIPS’06), pp. 153–160 Bengio Y, Lamblin P, Popovici D, Larochelle H (2007) Greedy layer-wise training of deep networks. In: Proceedings of the advances in neural information processing systems (NIPS’06), pp. 153–160
7.
Zurück zum Zitat Sermanet P, LeCun Y (2011) Traffic sign recognition with multi-scale convolutional networks. In: Proceedings of the 2011 international joint conference on neural networks (IJCNN) Sermanet P, LeCun Y (2011) Traffic sign recognition with multi-scale convolutional networks. In: Proceedings of the 2011 international joint conference on neural networks (IJCNN)
9.
Zurück zum Zitat Soleymanpour S, Sadr H, Beheshti H (2020) An efficient deep learning method for encrypted traffic classification on the web. In: 2020 6th International conference on web research (ICWR), IEEE, pp. 209–216 Soleymanpour S, Sadr H, Beheshti H (2020) An efficient deep learning method for encrypted traffic classification on the web. In: 2020 6th International conference on web research (ICWR), IEEE, pp. 209–216
10.
Zurück zum Zitat Sultana F, Sufian A, Dutta P (2018). Advancements in image classification using convolutional neural network. In: 2018 Fourth international conference on research in computational intelligence and communication networks (ICRCICN), IEEE, pp. 122–129 Sultana F, Sufian A, Dutta P (2018). Advancements in image classification using convolutional neural network. In: 2018 Fourth international conference on research in computational intelligence and communication networks (ICRCICN), IEEE, pp. 122–129
11.
Zurück zum Zitat Yadav SS, Jadhav SM (2019) Deep convolutional neural network based medical image classification for disease diagnosis. J Big Data 6(1):1–18CrossRef Yadav SS, Jadhav SM (2019) Deep convolutional neural network based medical image classification for disease diagnosis. J Big Data 6(1):1–18CrossRef
12.
Zurück zum Zitat Altan G, Yayik A, Kutlu Y (2021) Deep learning with ConvNet predicts imagery tasks through EEG. Neural Process Lett 53:1–16CrossRef Altan G, Yayik A, Kutlu Y (2021) Deep learning with ConvNet predicts imagery tasks through EEG. Neural Process Lett 53:1–16CrossRef
13.
14.
15.
Zurück zum Zitat Wang Y et al. (2016) CNNpack: Packing convolutional neural networks in the frequency domain. In: NIPS Wang Y et al. (2016) CNNpack: Packing convolutional neural networks in the frequency domain. In: NIPS
16.
Zurück zum Zitat Lee H, Groose R, Ranganath R, Ng A (2009) Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In: Proceedings of the 26th international conference on machine learning ( ICML09), pp. 609–616 Lee H, Groose R, Ranganath R, Ng A (2009) Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In: Proceedings of the 26th international conference on machine learning ( ICML09), pp. 609–616
17.
Zurück zum Zitat Kiranyaz S, Avci O, Abdeljaber O, Ince T, Gabbouj M, Inman DJ (2021) 1D convolutional neural networks and applications: a survey. Mech Syst Signal Process 151:107398CrossRef Kiranyaz S, Avci O, Abdeljaber O, Ince T, Gabbouj M, Inman DJ (2021) 1D convolutional neural networks and applications: a survey. Mech Syst Signal Process 151:107398CrossRef
18.
Zurück zum Zitat Borovykh A, Bohte S, Oosterlee CW (2017) Conditional time series forecasting with convolutional neural networks. arXiv preprint arXiv:1703.04691 Borovykh A, Bohte S, Oosterlee CW (2017) Conditional time series forecasting with convolutional neural networks. arXiv preprint arXiv:​1703.​04691
19.
Zurück zum Zitat Javedani SH et al (2019) Short-term load forecasting by using a combined method of convolutional neural networks and fuzzy time series. Energy 175:365–377CrossRef Javedani SH et al (2019) Short-term load forecasting by using a combined method of convolutional neural networks and fuzzy time series. Energy 175:365–377CrossRef
20.
Zurück zum Zitat Binkowski M, Marti G, Donnat P (2017) Autoregressive convolutional neural networks for asynchronous time series. arXiv:1703.04122 Binkowski M, Marti G, Donnat P (2017) Autoregressive convolutional neural networks for asynchronous time series. arXiv:​1703.​04122
22.
Zurück zum Zitat Wang Y (2017) A new concept using LSTM neural networks for dynamic system identification. In: 2017 American control conference (ACC), IEEE, pp 5324–5329 Wang Y (2017) A new concept using LSTM neural networks for dynamic system identification. In: 2017 American control conference (ACC), IEEE, pp 5324–5329
23.
Zurück zum Zitat Genc S (2017) Parametric system identification using deep convolutional neural networks. In: Proceedings of the 2017 international joint conference on neural networks (IJCNN17), pp. 2112–2119 Genc S (2017) Parametric system identification using deep convolutional neural networks. In: Proceedings of the 2017 international joint conference on neural networks (IJCNN17), pp. 2112–2119
24.
Zurück zum Zitat Lopez M, Yu W (2019) Impact of random weights on nonlinear system identification using convolutional neural networks. Inf Sci 477(1):1–14MathSciNet Lopez M, Yu W (2019) Impact of random weights on nonlinear system identification using convolutional neural networks. Inf Sci 477(1):1–14MathSciNet
25.
Zurück zum Zitat Kang Y, Chen S, Wang X, Cao Y (2018) Deep convolutional identifier for dynamic modeling and adaptive control of unmanned helicopter. IEEE Trans Neural Netw Learn Syst 30:524–538MathSciNetCrossRef Kang Y, Chen S, Wang X, Cao Y (2018) Deep convolutional identifier for dynamic modeling and adaptive control of unmanned helicopter. IEEE Trans Neural Netw Learn Syst 30:524–538MathSciNetCrossRef
27.
Zurück zum Zitat Xiong T et al (2015) Forecasting interval time series using a fully complex-valued RBF neural network with DPSO and PSO algorithms. Inf Sci 305:77–92CrossRef Xiong T et al (2015) Forecasting interval time series using a fully complex-valued RBF neural network with DPSO and PSO algorithms. Inf Sci 305:77–92CrossRef
28.
Zurück zum Zitat Dramsch J, Lüthje M, Christensen AN (2021) Complex-valued neural networks for machine learning on non-stationary physical data. Comput Geosci 146:104632 (Elsevier)CrossRef Dramsch J, Lüthje M, Christensen AN (2021) Complex-valued neural networks for machine learning on non-stationary physical data. Comput Geosci 146:104632 (Elsevier)CrossRef
30.
Zurück zum Zitat Benvenuto N, Piazza F (1992) On the complex backpropagation algorithm. IEEE Trans Signal Process 40(4):967–969CrossRef Benvenuto N, Piazza F (1992) On the complex backpropagation algorithm. IEEE Trans Signal Process 40(4):967–969CrossRef
31.
Zurück zum Zitat Hirose A, Yoshida S (2011) Comparison of complex-and real-valued feedforward neural networks in their generalization ability. In: International conference on neural information processing. Springer, Berlin, Heidelberg Hirose A, Yoshida S (2011) Comparison of complex-and real-valued feedforward neural networks in their generalization ability. In: International conference on neural information processing. Springer, Berlin, Heidelberg
32.
Zurück zum Zitat Hirose A (2003) Complex-valued neural networks: theories and applications, vol 5. World Scientific, SingaporeMATHCrossRef Hirose A (2003) Complex-valued neural networks: theories and applications, vol 5. World Scientific, SingaporeMATHCrossRef
33.
Zurück zum Zitat Quan Y, Chen Y, Shao Y, Teng H, Xu Y, Ji H (2021) Image denoising using complex-valued deep CNN. Pattern Recognit 111:107639CrossRef Quan Y, Chen Y, Shao Y, Teng H, Xu Y, Ji H (2021) Image denoising using complex-valued deep CNN. Pattern Recognit 111:107639CrossRef
34.
Zurück zum Zitat Zhang Z, Wang H, Xu F, Jin YQ (2017) Complex-valued convolutional neural network and its application in polarimetric SAR image classification. IEEE Trans Geosci Remote Sens 55(12):7177–7188CrossRef Zhang Z, Wang H, Xu F, Jin YQ (2017) Complex-valued convolutional neural network and its application in polarimetric SAR image classification. IEEE Trans Geosci Remote Sens 55(12):7177–7188CrossRef
35.
Zurück zum Zitat Meyer M, Kuschk G, Tomforde, S (2020) Complex-valued convolutional neural networks for automotive scene classification based on range-beam-doppler tensors. In: 2020 IEEE 23rd International conference on intelligent transportation systems (ITSC), IEEE, pp. 1–6 Meyer M, Kuschk G, Tomforde, S (2020) Complex-valued convolutional neural networks for automotive scene classification based on range-beam-doppler tensors. In: 2020 IEEE 23rd International conference on intelligent transportation systems (ITSC), IEEE, pp. 1–6
36.
Zurück zum Zitat Yao X, Shi X, Zhou F (2020) Human activities classification based on complex-value convolutional neural network. IEEE Sens J 20(13):7169–7180CrossRef Yao X, Shi X, Zhou F (2020) Human activities classification based on complex-value convolutional neural network. IEEE Sens J 20(13):7169–7180CrossRef
37.
Zurück zum Zitat Zimmermann HG et al (2011) Comparison of the complex valued and real valued neural networks trained with gradient descent and random search algorithms. In: Proc. of ESANN 2011 Zimmermann HG et al (2011) Comparison of the complex valued and real valued neural networks trained with gradient descent and random search algorithms. In: Proc. of ESANN 2011
38.
Zurück zum Zitat Narendra K, Parthasarathy K (1990) Identification and control of dynamical systems using neural networks. IEEE Trans Neural Netw 1(1):4–27CrossRef Narendra K, Parthasarathy K (1990) Identification and control of dynamical systems using neural networks. IEEE Trans Neural Netw 1(1):4–27CrossRef
39.
Zurück zum Zitat Ruta D, Gabrys B (2007) Neural network ensembles for time series prediction. Int Joint Conf Neural Netw 2007:1204–1209CrossRef Ruta D, Gabrys B (2007) Neural network ensembles for time series prediction. Int Joint Conf Neural Netw 2007:1204–1209CrossRef
40.
Zurück zum Zitat Gu H, Qing G, Wang Y, Hong S, Gacanin H, Adachi F (2020) Deep complex-valued convolutional neural network for drone recognition based on RF fingerprinting Gu H, Qing G, Wang Y, Hong S, Gacanin H, Adachi F (2020) Deep complex-valued convolutional neural network for drone recognition based on RF fingerprinting
41.
Zurück zum Zitat Tan X, Li M, Zhang P, Wu Y, Song W (2019) Complex-valued 3-D convolutional neural network for PolSAR image classification. IEEE Geosci Remote Sens Lett 17(6):1022–1026CrossRef Tan X, Li M, Zhang P, Wu Y, Song W (2019) Complex-valued 3-D convolutional neural network for PolSAR image classification. IEEE Geosci Remote Sens Lett 17(6):1022–1026CrossRef
42.
Zurück zum Zitat Hongyo R, Egashira Y, Yamaguchi K (2018) Deep neural network based predistorter with ReLU activation for Doherty power amplifiers. Asia-Pacific Microw Conf (APMC) 2018:959–961 Hongyo R, Egashira Y, Yamaguchi K (2018) Deep neural network based predistorter with ReLU activation for Doherty power amplifiers. Asia-Pacific Microw Conf (APMC) 2018:959–961
43.
Zurück zum Zitat James G, Burley D (2002) Matematicas avanzadas para ingenieria. Pearson Educacion, London James G, Burley D (2002) Matematicas avanzadas para ingenieria. Pearson Educacion, London
44.
Zurück zum Zitat Bergstra J, Bengio Y (2011) Random search for hyper-parameter optimization. J Mach Learn Res 13:281–305MathSciNetMATH Bergstra J, Bengio Y (2011) Random search for hyper-parameter optimization. J Mach Learn Res 13:281–305MathSciNetMATH
45.
Zurück zum Zitat Box G, Jenkins G, Reinsel G (2008) Time series analysis: forecasting and control, 4th edn. Wiley, HobokenMATHCrossRef Box G, Jenkins G, Reinsel G (2008) Time series analysis: forecasting and control, 4th edn. Wiley, HobokenMATHCrossRef
46.
Zurück zum Zitat Schoukens J, Suykens J, Ljung L (2009) Wiener-Hammerstein benchmark. In: Proceedings of the 5th IFAC symposium on system identification Schoukens J, Suykens J, Ljung L (2009) Wiener-Hammerstein benchmark. In: Proceedings of the 5th IFAC symposium on system identification
47.
Zurück zum Zitat Ustundag BB, Kulaglic A (2020) High-performance time series prediction with predictive error compensated wavelet neural networks. IEEE Access 8:210532–210541CrossRef Ustundag BB, Kulaglic A (2020) High-performance time series prediction with predictive error compensated wavelet neural networks. IEEE Access 8:210532–210541CrossRef
48.
Zurück zum Zitat George K, Mutalik P (2017) A multiple model approach to time-series prediction using an online sequential learning algorithm. IEEE Trans Syst, Man, Cybern: Syst 49(5):976–990CrossRef George K, Mutalik P (2017) A multiple model approach to time-series prediction using an online sequential learning algorithm. IEEE Trans Syst, Man, Cybern: Syst 49(5):976–990CrossRef
49.
Zurück zum Zitat Gonzalez J, Yu W (2018) Non-linear system modeling using LSTM neural networks. IFAC-PapersOnLine 51(13):485–489CrossRef Gonzalez J, Yu W (2018) Non-linear system modeling using LSTM neural networks. IFAC-PapersOnLine 51(13):485–489CrossRef
50.
Zurück zum Zitat Shaikh MAH, Barbà K (2019) Wiener Hammerstein system identification: a fast approach through spearman correlation. IEEE Trans Instrum Measurement 68(5):1628–1636CrossRef Shaikh MAH, Barbà K (2019) Wiener Hammerstein system identification: a fast approach through spearman correlation. IEEE Trans Instrum Measurement 68(5):1628–1636CrossRef
51.
Zurück zum Zitat Marconato A, Schoukens J (2009) Identification of Wiener-Hammerstein benchmark data by means of support vector machines. IFAC Proc Vol 42(10):816–819CrossRef Marconato A, Schoukens J (2009) Identification of Wiener-Hammerstein benchmark data by means of support vector machines. IFAC Proc Vol 42(10):816–819CrossRef
52.
Zurück zum Zitat Paduart J, Lauwers L, Pintelon R, Schoukens J (2012) Identification of a WienerHammerstein system using the polynomial nonlinear state space approach. Control Eng Pract 20(11):1133–1139CrossRef Paduart J, Lauwers L, Pintelon R, Schoukens J (2012) Identification of a WienerHammerstein system using the polynomial nonlinear state space approach. Control Eng Pract 20(11):1133–1139CrossRef
Metadaten
Titel
Complex Valued Deep Neural Networks for Nonlinear System Modeling
verfasst von
Mario Lopez-Pacheco
Wen Yu
Publikationsdatum
23.09.2021
Verlag
Springer US
Erschienen in
Neural Processing Letters / Ausgabe 1/2022
Print ISSN: 1370-4621
Elektronische ISSN: 1573-773X
DOI
https://doi.org/10.1007/s11063-021-10644-1

Weitere Artikel der Ausgabe 1/2022

Neural Processing Letters 1/2022 Zur Ausgabe

Neuer Inhalt