Skip to main content
main-content
Top

Hint

Swipe to navigate through the articles of this issue

24-09-2022

An Outlier-Robust Growing Local Model Network for Recursive System Identification

Authors: Jéssyca A. Bessa, Guilherme A. Barreto, Ajalmar R. Rocha-Neto

Published in: Neural Processing Letters

Login to get access
share
SHARE

Abstract

In this paper, we develop a self-growing variant of the local model network (LMN) for recursive dynamical system identification. The proposed model has the following features: growing online structure, fast recursive updating rules, better memory use (no storage of covariance matrices is required), and outlier-robustness. In this regard, efficiency in performance and simplicity of implementation are the essential qualities of the proposed approach. The proposed growing version of the LMN model results from a synergistic amalgamation of two simple but powerful ideas. For this purpose, we adapt the neuron insertion strategy of the resource-allocating network to LMN model, and replaces the standard OLS rule for parameter estimation with outlier-robust recursive rules. A comprehensive evaluation involving three SISO and one MIMO benchmarking data sets corroborates the proposed approach’s superior predictive performance in outlier-contaminated scenarios compared to fixed-size LMN-based models.
Literature
1.
go back to reference Angelov PP, Filev DP (2004) An approach to online identification of Takagi-Sugeno fuzzy models. IEEE Trans Syst Man Cybern Part B 34(1):484–497 CrossRef Angelov PP, Filev DP (2004) An approach to online identification of Takagi-Sugeno fuzzy models. IEEE Trans Syst Man Cybern Part B 34(1):484–497 CrossRef
2.
go back to reference Barreto GA, Araújo AFR (2004) Identification and control of dynamical systems using the self-organizing map. IEEE Trans Neural Netw 15(5):1244–1259 CrossRef Barreto GA, Araújo AFR (2004) Identification and control of dynamical systems using the self-organizing map. IEEE Trans Neural Netw 15(5):1244–1259 CrossRef
3.
go back to reference Barreto GA, Souza LGM (2016) Novel approaches for parameter estimation of local linear models for dynamical system identification. Appl Intell 44(1):149–165 CrossRef Barreto GA, Souza LGM (2016) Novel approaches for parameter estimation of local linear models for dynamical system identification. Appl Intell 44(1):149–165 CrossRef
4.
go back to reference Belz J, Münker T, Heinz TO, Kampmann G, Nelles O (2017) Automatic modeling with local model networks for benchmark processes. IFAC-PapersOnLine 50(1):470–475 CrossRef Belz J, Münker T, Heinz TO, Kampmann G, Nelles O (2017) Automatic modeling with local model networks for benchmark processes. IFAC-PapersOnLine 50(1):470–475 CrossRef
5.
go back to reference Bessa JA, Barreto GA (2019) Recursive system identification using outlier-robust local models. IFAC-PapersOnLine 52(1):436–441 CrossRef Bessa JA, Barreto GA (2019) Recursive system identification using outlier-robust local models. IFAC-PapersOnLine 52(1):436–441 CrossRef
6.
go back to reference Billings SA (2013) Nonlinear system identification: NARMAX methods in the time, frequency, and spatio-temporal domains. Wiley, London MATHCrossRef Billings SA (2013) Nonlinear system identification: NARMAX methods in the time, frequency, and spatio-temporal domains. Wiley, London MATHCrossRef
7.
go back to reference Bittanti S, Picci G (1996) Identification, adaptation, learning: the science of learning models from data Bittanti S, Picci G (1996) Identification, adaptation, learning: the science of learning models from data
8.
go back to reference Blažič S, Škrjanc I (2020) Incremental fuzzy C-regression clustering from streaming data for local-model-network identification. IEEE Trans Fuzzy Syst 28(4):758–767 CrossRef Blažič S, Škrjanc I (2020) Incremental fuzzy C-regression clustering from streaming data for local-model-network identification. IEEE Trans Fuzzy Syst 28(4):758–767 CrossRef
9.
go back to reference Brunton SL, Kutz JN (2019) Data-driven science and engineering: machine learning, dynamical systems, and control. Cambridge Press, Cambridge Brunton SL, Kutz JN (2019) Data-driven science and engineering: machine learning, dynamical systems, and control. Cambridge Press, Cambridge
10.
go back to reference Chan S-C, Zhou Y (2010) On the performance analysis of the least mean M-estimate and normalized least mean M-estimate algorithms with gaussian inputs and additive gaussian and contaminated Gaussian noises. J Signal Process Syst 60(1):81–103 CrossRef Chan S-C, Zhou Y (2010) On the performance analysis of the least mean M-estimate and normalized least mean M-estimate algorithms with gaussian inputs and additive gaussian and contaminated Gaussian noises. J Signal Process Syst 60(1):81–103 CrossRef
11.
go back to reference Chen S, Billings SA, Cowan CFN, Grant PM (1990) Non-linear systems identification using radial basis functions. Int J Syst Sci 21(12):2513–2539 MATHCrossRef Chen S, Billings SA, Cowan CFN, Grant PM (1990) Non-linear systems identification using radial basis functions. Int J Syst Sci 21(12):2513–2539 MATHCrossRef
12.
go back to reference Chou CT, Maciejowski JM (1997) System identification using balanced parametrizations. IEEE Trans Autom Control 42(7):956–974 MATHCrossRef Chou CT, Maciejowski JM (1997) System identification using balanced parametrizations. IEEE Trans Autom Control 42(7):956–974 MATHCrossRef
13.
go back to reference Chuang C-C (2007) Fuzzy weighted support vector regression with a fuzzy partition. IEEE Trans Syst Man Cybern Part B 37(3):630–640 CrossRef Chuang C-C (2007) Fuzzy weighted support vector regression with a fuzzy partition. IEEE Trans Syst Man Cybern Part B 37(3):630–640 CrossRef
14.
go back to reference Costa TV, Fileti AMF, Oliveira-Lopes LC, Silva FV (2015) Experimental assessment and design of multiple model predictive control based on local model networks for industrial processes. Evol Syst 6(4):243–253 CrossRef Costa TV, Fileti AMF, Oliveira-Lopes LC, Silva FV (2015) Experimental assessment and design of multiple model predictive control based on local model networks for industrial processes. Evol Syst 6(4):243–253 CrossRef
15.
go back to reference Foss BA, Johansen TA (1993) , On local and fuzzy modelling. In: Third International Conference on Industrial Fuzzy Control and Intelligent Systems (IFIS’93), pp 80–87 Foss BA, Johansen TA (1993) , On local and fuzzy modelling. In: Third International Conference on Industrial Fuzzy Control and Intelligent Systems (IFIS’93), pp 80–87
16.
go back to reference Fritzke B (1995) A growing neural gas network learns topologies. In: Advances in neural information processing systems 7. MIT Press, pp 625–632 Fritzke B (1995) A growing neural gas network learns topologies. In: Advances in neural information processing systems 7. MIT Press, pp 625–632
17.
go back to reference Fritzke B (1995) Incremental learning of local linear mappings. In: Proceedings of the international conference on artificial neural networks (ICANN’95’), pp. 217–222 Fritzke B (1995) Incremental learning of local linear mappings. In: Proceedings of the international conference on artificial neural networks (ICANN’95’), pp. 217–222
18.
go back to reference Fuangkhon P (2014) An incremental learning preprocessor for feed-forward neural network. Artif Intell Rev 41(2):183–210 CrossRef Fuangkhon P (2014) An incremental learning preprocessor for feed-forward neural network. Artif Intell Rev 41(2):183–210 CrossRef
19.
go back to reference Hampel FR, Ronchetti EM, Rousseeuw PJ, Stahel WA (2011) Robust statistics: the approach based on influence functions, vol. 114. Wiley Hampel FR, Ronchetti EM, Rousseeuw PJ, Stahel WA (2011) Robust statistics: the approach based on influence functions, vol. 114. Wiley
20.
go back to reference Han H, Chen Q, Qiao J (2010) Research on an online self-organizing radial basis function neural network. Neural Comput Appl 19:667–676 CrossRef Han H, Chen Q, Qiao J (2010) Research on an online self-organizing radial basis function neural network. Neural Comput Appl 19:667–676 CrossRef
21.
go back to reference Hassibi B, Sayed AH, Kailath T (1996) H \(_\infty \) optimality of the LMS algorithm. IEEE Trans Signal Process 44(2):267–280 CrossRef Hassibi B, Sayed AH, Kailath T (1996) H \(_\infty \) optimality of the LMS algorithm. IEEE Trans Signal Process 44(2):267–280 CrossRef
22.
go back to reference He H, Chen S, Li K, Xu X (2011) Incremental learning from stream data. IEEE Trans Neural Netw 22(12):1901–1914 CrossRef He H, Chen S, Li K, Xu X (2011) Incremental learning from stream data. IEEE Trans Neural Netw 22(12):1901–1914 CrossRef
24.
go back to reference Jain S, Bandyopadhyay S (2017) Resource allocation network for segregated targeting problems with dedicated sources. Ind Eng Chem Res 56(46):13831–13843 CrossRef Jain S, Bandyopadhyay S (2017) Resource allocation network for segregated targeting problems with dedicated sources. Ind Eng Chem Res 56(46):13831–13843 CrossRef
25.
go back to reference Johansen TA, Foss BA (1992) A NARMAX model representation for adaptive control based on local models. Model Identif Control 13(1):25 CrossRef Johansen TA, Foss BA (1992) A NARMAX model representation for adaptive control based on local models. Model Identif Control 13(1):25 CrossRef
26.
go back to reference Jones RD, Lee YC, Barnes CW, Flake GW, Lee K, Lewis PS, Qian S (1990) Function approximation and time series prediction with neural networks. In: Proceedings of the 1990 international joint conference on neural networks (IJCNN’1990)’, pp 649–665 Jones RD, Lee YC, Barnes CW, Flake GW, Lee K, Lewis PS, Qian S (1990) Function approximation and time series prediction with neural networks. In: Proceedings of the 1990 international joint conference on neural networks (IJCNN’1990)’, pp 649–665
27.
go back to reference Kiumarsi B, Lewis FL, Levine SD (2015) Optimal control of nonlinear discrete time-varying systems using a new neural network approximation structure. Neurocomputing 156:157–165 Kiumarsi B, Lewis FL, Levine SD (2015) Optimal control of nonlinear discrete time-varying systems using a new neural network approximation structure. Neurocomputing 156:157–165
28.
go back to reference Kohonen T (2013) Essentials of the self-organizing map. Neural Netw 37:52–65 CrossRef Kohonen T (2013) Essentials of the self-organizing map. Neural Netw 37:52–65 CrossRef
29.
go back to reference König O, Hametner C, Prochart G, Jakubek S (2014) Battery emulation for power-HIL using local model networks and robust impedance control. IEEE Trans Ind Electron 61(2):943–955 CrossRef König O, Hametner C, Prochart G, Jakubek S (2014) Battery emulation for power-HIL using local model networks and robust impedance control. IEEE Trans Ind Electron 61(2):943–955 CrossRef
30.
go back to reference Kovačević B, Banjac Z, Kovačević IK (2016) Robust adaptive filtering using recursive weighted least squares with combined scale and variable forgetting factors. EURASIP J Adv Signal Process 37:1–22 Kovačević B, Banjac Z, Kovačević IK (2016) Robust adaptive filtering using recursive weighted least squares with combined scale and variable forgetting factors. EURASIP J Adv Signal Process 37:1–22
31.
go back to reference Liu J, Djurdjanovic D, Marko K, Ni J (2009) Growing structure multiple model systems for anomaly detection and fault diagnosis. J Dyn Syst Meas Contr 131(5):1–13 CrossRef Liu J, Djurdjanovic D, Marko K, Ni J (2009) Growing structure multiple model systems for anomaly detection and fault diagnosis. J Dyn Syst Meas Contr 131(5):1–13 CrossRef
32.
go back to reference Ljung L (1999) System identification: theory for the user, 2nd edn. Prentice Hall Ljung L (1999) System identification: theory for the user, 2nd edn. Prentice Hall
33.
go back to reference Lu T, Pan L, Jiangs J, Zhang Y, Xiong Z (2017) Dlml: deep linear mappings learning for face super-resolution with nonlocal-patch. In: 2017 IEEE international conference on multimedia and expo (ICME), IEEE, pp 1362–1367 Lu T, Pan L, Jiangs J, Zhang Y, Xiong Z (2017) Dlml: deep linear mappings learning for face super-resolution with nonlocal-patch. In: 2017 IEEE international conference on multimedia and expo (ICME), IEEE, pp 1362–1367
34.
go back to reference Lv C, Zhang T, Ma F, Yue D (2018) A very short-term online forecasting model for photovoltaic power based on two-stage resource allocation network. In: 2018 International joint conference on neural networks (IJCNN), IEEE, pp 1–6 Lv C, Zhang T, Ma F, Yue D (2018) A very short-term online forecasting model for photovoltaic power based on two-stage resource allocation network. In: 2018 International joint conference on neural networks (IJCNN), IEEE, pp 1–6
35.
go back to reference Mahanand BS, Suresh S, Sundararajan N, Kumar MA (2012) Identification of brain regions responsible for Alzheimer’s disease using a self-adaptive resource allocation network. Neural Netw 32:313–322 CrossRef Mahanand BS, Suresh S, Sundararajan N, Kumar MA (2012) Identification of brain regions responsible for Alzheimer’s disease using a self-adaptive resource allocation network. Neural Netw 32:313–322 CrossRef
36.
go back to reference Maier CC, Schirrer A, Kozek M (2018) Real-time capable nonlinear pantograph models using local model networks in state-space configuration. Mechatronics 50:292–302 CrossRef Maier CC, Schirrer A, Kozek M (2018) Real-time capable nonlinear pantograph models using local model networks in state-space configuration. Mechatronics 50:292–302 CrossRef
37.
go back to reference Mandic DP, Kanna S, Constantinides AG (2015) On the intrinsic relationship between the least mean square and Kalman filters. IEEE Signal Process Mag 32(6):117–122 CrossRef Mandic DP, Kanna S, Constantinides AG (2015) On the intrinsic relationship between the least mean square and Kalman filters. IEEE Signal Process Mag 32(6):117–122 CrossRef
38.
go back to reference Martinetz T (1993) , Competitive hebbian learning rule forms perfectly topology preserving maps. In: ICANN’93, Springer, pp 427–434 Martinetz T (1993) , Competitive hebbian learning rule forms perfectly topology preserving maps. In: ICANN’93, Springer, pp 427–434
39.
go back to reference Mattos CLC, Dai Z, Damianou A, Barreto GA, Lawrence ND (2017) Deep recurrent gaussian processes for outlier-robust system identification. J Process Control 60:82–94 CrossRef Mattos CLC, Dai Z, Damianou A, Barreto GA, Lawrence ND (2017) Deep recurrent gaussian processes for outlier-robust system identification. J Process Control 60:82–94 CrossRef
40.
go back to reference Moody J, Darken CJ (1989) Fast learning in networks of locally-tuned processing units. Neural Comput 1(2):281–294 CrossRef Moody J, Darken CJ (1989) Fast learning in networks of locally-tuned processing units. Neural Comput 1(2):281–294 CrossRef
41.
go back to reference Moshou D, Ramon H (1997) Extended self-organizing maps with local linear mappings for function approximation and system identification. In: Proceedings of the 1st workshop on self-organizing maps (WSOM’97), pp 1–6 Moshou D, Ramon H (1997) Extended self-organizing maps with local linear mappings for function approximation and system identification. In: Proceedings of the 1st workshop on self-organizing maps (WSOM’97), pp 1–6
42.
go back to reference Münker T, Heinz TO, Nelles O (2017) , Hierarchical model predictive control for local model networks. In: Proceedings of the American Control Conference (ACC’2017), pp 5026–5031 Münker T, Heinz TO, Nelles O (2017) , Hierarchical model predictive control for local model networks. In: Proceedings of the American Control Conference (ACC’2017), pp 5026–5031
43.
go back to reference Narendra K, Parthasarathy K (1990) Identification and control of dynamical systems using neural networks. IEEE Trans Neural Netw 1(1):4–27 CrossRef Narendra K, Parthasarathy K (1990) Identification and control of dynamical systems using neural networks. IEEE Trans Neural Netw 1(1):4–27 CrossRef
44.
go back to reference Pai PS, Nagabhushana T, Rao PR (2001) Tool wear estimation using resource allocation network. Int J Mach Tools Manuf 41(5):673–685 CrossRef Pai PS, Nagabhushana T, Rao PR (2001) Tool wear estimation using resource allocation network. Int J Mach Tools Manuf 41(5):673–685 CrossRef
46.
go back to reference Poggio T, Girosi F (1990) Networks for approximation and learning. Proc IEEE 78(9):1484–1487 MATHCrossRef Poggio T, Girosi F (1990) Networks for approximation and learning. Proc IEEE 78(9):1484–1487 MATHCrossRef
47.
go back to reference Prasad G, Swidenbank E, Hogg BW (1998) A local model networks based multivariable long-range predictive control strategy for thermal power plants. Automatica 34(10):1185–1204 MATHCrossRef Prasad G, Swidenbank E, Hogg BW (1998) A local model networks based multivariable long-range predictive control strategy for thermal power plants. Automatica 34(10):1185–1204 MATHCrossRef
48.
go back to reference Ribeiro AH, Aguirre LA (2018) Parallel training considered harmful?: Comparing series-parallel and parallel feedforward network training. Neurocomputing 316:222–231 CrossRef Ribeiro AH, Aguirre LA (2018) Parallel training considered harmful?: Comparing series-parallel and parallel feedforward network training. Neurocomputing 316:222–231 CrossRef
49.
go back to reference Schoukens J, Nemeth JG, Crama P, Rolain Y, Pintelon R (2003) Fast approximate identification of nonlinear systems. Automatica 39(7):1267–1274 MathSciNetMATHCrossRef Schoukens J, Nemeth JG, Crama P, Rolain Y, Pintelon R (2003) Fast approximate identification of nonlinear systems. Automatica 39(7):1267–1274 MathSciNetMATHCrossRef
50.
go back to reference Sjöberg J, Zhang Q, Ljung L, Benveniste A, Delyon B, Glorennec P-Y, Hjalmarsson H, Juditsky A (1995) Nonlinear black-box modeling in system identification: a unified overview. Automatica 31(12):1691–1724 MathSciNetMATHCrossRef Sjöberg J, Zhang Q, Ljung L, Benveniste A, Delyon B, Glorennec P-Y, Hjalmarsson H, Juditsky A (1995) Nonlinear black-box modeling in system identification: a unified overview. Automatica 31(12):1691–1724 MathSciNetMATHCrossRef
51.
go back to reference Soong TT (2004) Fundamentals of probability and statistics for engineers. Wiley, London MATH Soong TT (2004) Fundamentals of probability and statistics for engineers. Wiley, London MATH
52.
go back to reference Stokbro K, Umberger DK, Hertz JA (1990) Exploiting neurons with localized receptive fields to learn chaos. Complex Syst 4(3):603–622 MATH Stokbro K, Umberger DK, Hertz JA (1990) Exploiting neurons with localized receptive fields to learn chaos. Complex Syst 4(3):603–622 MATH
53.
go back to reference Suresh S, Dong K, Kim HJ (2010) A sequential learning algorithm for self-adaptive resource allocation network classifier. Neurocomputing 73(16–18):3012–3019 CrossRef Suresh S, Dong K, Kim HJ (2010) A sequential learning algorithm for self-adaptive resource allocation network classifier. Neurocomputing 73(16–18):3012–3019 CrossRef
54.
go back to reference Takagi T, Sugeno M (1985) Fuzzy identification of systems and its applications to modelling and control. IEEE Trans Syst Man Cybern 15(1):116–132 MATHCrossRef Takagi T, Sugeno M (1985) Fuzzy identification of systems and its applications to modelling and control. IEEE Trans Syst Man Cybern 15(1):116–132 MATHCrossRef
55.
go back to reference Vachkov G (2004) , Growing model algorithm for process identification based on neural-gas learning and local linear mapping. In: Fourth international conference on hybrid intelligent systems (HIS’04), IEEE, pp 222–227 Vachkov G (2004) , Growing model algorithm for process identification based on neural-gas learning and local linear mapping. In: Fourth international conference on hybrid intelligent systems (HIS’04), IEEE, pp 222–227
56.
go back to reference Walter J, Ritter H, Schulten K (1990) , Nonlinear prediction with self-organizing maps. In: Proceedings of the IEEE international joint conference on neural networks (IJCNN’90), pp 589–594 Walter J, Ritter H, Schulten K (1990) , Nonlinear prediction with self-organizing maps. In: Proceedings of the IEEE international joint conference on neural networks (IJCNN’90), pp 589–594
57.
go back to reference Wang Q, Wang X, Sun B (2019) Resource allocation network and PID control based on automated guided vehicles. Acad J Manuf Eng 17(2) Wang Q, Wang X, Sun B (2019) Resource allocation network and PID control based on automated guided vehicles. Acad J Manuf Eng 17(2)
58.
go back to reference Widrow B (2005) Thinking about thinking: the discovery of the LMS algorithm. IEEE Signal Process Mag 22(1):100–106 CrossRef Widrow B (2005) Thinking about thinking: the discovery of the LMS algorithm. IEEE Signal Process Mag 22(1):100–106 CrossRef
59.
go back to reference Widrow B, Lehr MA (1990) 30 years of adaptive neural networks: perceptron, madaline and backpropagation. Proc IEEE 78(9):1415–1442 CrossRef Widrow B, Lehr MA (1990) 30 years of adaptive neural networks: perceptron, madaline and backpropagation. Proc IEEE 78(9):1415–1442 CrossRef
60.
go back to reference Yan L, Sundararajan N, Saratchandran P (2000) Nonlinear system identification using Lyapunov based fully tuned dynamic RBF networks. Neural Process Lett 12(3):291–303 MATHCrossRef Yan L, Sundararajan N, Saratchandran P (2000) Nonlinear system identification using Lyapunov based fully tuned dynamic RBF networks. Neural Process Lett 12(3):291–303 MATHCrossRef
61.
go back to reference Zhao Z-L, Qiu Z-C, Zhang X-M, Han J-D (2016) Vibration control of a pneumatic driven piezoelectric flexible manipulator using self-organizing map based multiple models. Mech Syst Signal Process 70:345–372 CrossRef Zhao Z-L, Qiu Z-C, Zhang X-M, Han J-D (2016) Vibration control of a pneumatic driven piezoelectric flexible manipulator using self-organizing map based multiple models. Mech Syst Signal Process 70:345–372 CrossRef
62.
go back to reference Zou Y, Chan S-C, Ng T-S (2000) Least mean M-estimate algorithms for robust adaptive filtering in impulse noise. IEEE Trans Circuits Syst II: Analog Digit Signal Process 47(12):1564–1569 CrossRef Zou Y, Chan S-C, Ng T-S (2000) Least mean M-estimate algorithms for robust adaptive filtering in impulse noise. IEEE Trans Circuits Syst II: Analog Digit Signal Process 47(12):1564–1569 CrossRef
63.
go back to reference Zou Y, Chan SC, Ng TS (2001) Robust M-estimate adaptive filtering. IEE Proc Vis Image Signal Process 148(4):289–294 CrossRef Zou Y, Chan SC, Ng TS (2001) Robust M-estimate adaptive filtering. IEE Proc Vis Image Signal Process 148(4):289–294 CrossRef
Metadata
Title
An Outlier-Robust Growing Local Model Network for Recursive System Identification
Authors
Jéssyca A. Bessa
Guilherme A. Barreto
Ajalmar R. Rocha-Neto
Publication date
24-09-2022
Publisher
Springer US
Published in
Neural Processing Letters
Print ISSN: 1370-4621
Electronic ISSN: 1573-773X
DOI
https://doi.org/10.1007/s11063-022-11040-z