Abstract
This chapter continues the discussion of local linear neuro-fuzzy models and extends them to local model networks where the local models can be arbitrary and the fuzzy logic interpretation vanishes in the background. A couple of attractive key features of these model architectures are analyzed in this chapter. Two of them shall be highlighted here: The separation of the inputs for rule premises and for the rule consequents (local models) allows to partly overcome the curse of dimensionality because it is often possible to choose the premise input space of low dimension even if the local models are high-dimensional. This feature also will play a crucial role when dealing with dynamic models in Part C. Another nice characteristic is due to the local nature of this model architecture. It allows for extremely robust online learning without unlearning in regions where no new data arrives. Furthermore, the extension to axis-oblique tree construction is discussed and analyzed in great detail. A new algorithm called hierarchical local model tree (HILOMOT) is introduced and compared to LOLIMOT. This opens the door to solving even higher-dimensional problems with this kind of model architectures. On the basis of HILOMOT, a new design of experiments/active learning scheme is proposed that exploits the advantages of the model structure to the fullest. It has been successfully applied to multiple real-world problems that are covered in Part D.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Strictly speaking the interpolation region of the normalized Gaussians are infinitely large. However, for the degree of accuracy required in any practical consideration, it is virtually equivalent to the interpolation interval [1∕3, 2∕3] of the triangular validity functions.
- 2.
The term “learning” is used if the model possesses a memory in the sense that it does not forget previously learned relationships when the operating conditions change. Thus, here “learning” implies “adaptive nonlinear” plus a mechanism against arbitrary forgetting, e.g., locality.
- 3.
Note that this is not possible for the very first split since the root possesses no parent and it significantly increases the risk of converging to a local optimum.
- 4.
Will be normalized according to (14.39) anyway.
- 5.
Not exactly, see the paragraph about optimism in Sect. 7.2.3.
- 6.
Otherwise a superior white box model could be used.
- 7.
- 8.
The widths can be chosen individually for each input dimension but are identical for all kernels.
- 9.
- 10.
This is quite unlikely to happen but for a 1D example due to the suboptimal nature of incrementally building, the tree can occur sometimes.
- 11.
References
Abou El Ela, A.: Sensorarme Methoden zur Bearbeitung komplexer Werkstücke mit Industrierobotern. Ph.D. Thesis, TU Darmstadt, Reihe 8: Mess-, Steuerungs- und Regelungstechnik, Nr. 824. VDI-Verlag, Düsseldorf (2000)
Aggarwal, C.C., Hinneburg, A., Keim, D.A.: On the surprising behavior of distance metrics in high dimensional space. In: International Conference on Database Theory, pp. 420–434. Springer (2001)
Aleksovski, D., Kocijan, J., Dzeroski, S.: Model tree ensembles for the identification of multiple-output systems. In: Control Conference (ECC), 2014 European, pp. 750–755. IEEE (2014)
Alessio, A., Bemporad, A.: A survey on explicit model predictive control. In: Nonlinear Model Predictive Control, pp. 345–369. Springer (2009)
Atkeson, C.G., Moore, A.W., Schaal, S.: Locally weighted learning. Artif. Intell. Rev. 11(1–5), 11–73 (1997)
Auer, P.: Using confidence bounds for exploitation-exploration trade-offs. J. Mach. Learn. Res. 3, 397–422 (2002)
Babuška, R.: Fuzzy Modeling and Identification. Ph.D. thesis, Dept. of Control Engineering, Delft University of Technology, Delft, The Netherlands (1996)
Babuška, R., Fantuzzi, C., Kaymak, U., Verbruggen, H.B.: Improved inference for Takagi-Sugeno models. In: IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), pp. 701–706, New Orleans, USA (1996)
Babuška, R., Verbruggen, H.B.: An overview of fuzzy modeling for control. Control Eng. Pract. 4(11), 1593–1606 (1996)
Babuška, R., Verbruggen, H.B.: Fuzzy set methods for local modelling and identification. In: Murray-Smith, R., Johansen, T.A. (eds.) Multiple Model Approaches to Modelling and Control, chapter 2, pp. 75–100. Taylor & Francis, London (1997)
Belz, J., Nelles, O.: Honda project: input selection with local linear models. Technical report, University of Siegen (2015)
Bemporad, A., Borrelli, F., Morari, M., et al.: Model predictive control based on linear programming˜ the explicit solution. IEEE Trans. Autom. Control 47(12), 1974–1985 (2002)
Benner, M.J., Tushman, M.L.: Exploitation, exploration, and process management: the productivity dilemma revisited. Acad. Manag Rev. 28(2), 238–256 (2003)
Biswas, P., Grieder, P., Löfberg, J., Morari, M.: A survey on stability analysis of discrete-time piecewise affine systems. IFAC Proc. 38(1), 283–294 (2005)
Brandimarte, P.: Low-discrepancy sequences. In: Handbook in Monte Carlo Simulation: Applications in Financial Engineering, Risk Management, and Economics, pp. 379–401 (2014)
Breiman, L.: Hinging hyperplanes for regression, classification, and function approximation. IEEE Trans. Inf. Theory 39(3), 999–1013 (1993)
Breiman, L.: Bagging predictors. Mach. Learn. 24(2), 123–140 (1996)
Breiman, L.: Out-of-bag estimation. Technical report, Citeseer (1996)
Carpenter, G., Grossberg, S.: The ART of adaptive pattern recognition by a self-organizing neural network. IEEE Comput. 21(3), 77–88 (1988)
Cohn, D.: Neural network exploration using optimal experiment design. In: Cowan, J.D., Tesauro, G., Alspector, J. (eds.) Advances in Neural Information Processing Systems, vol. 6, pp. 1071–1083. Morgan Kaufmann, San Francisco (1994)
Cohn, D., Atlas, L., Ladner, R.: Training connectionist networks with queries and selective sampling. In: Touretzky, D.S. (ed.) Advances in Neural Information Processing Systems, vol. 2. Morgan Kaufmann, San Mateo (1990)
Cohn, D., Ghahramani, Z., Jordan, M.I.: Active learning with mixture models. In: Murray-Smith, R., Johansen, T.A. (eds.) Multiple Model Approaches to Modelling and Control, chapter 6, pp. 167–184. Taylor & Francis, London (1997)
Didcock, N., Jakubek, S., Kögeler, H.-M.: Regularisation methods for neural network model averaging. Eng. Appl. Artif. Intell. 41, 128–138 (2015)
Dietterich, T.G.: Ensemble methods in machine learning. In: International Workshop on Multiple Classifier Systems, pp. 1–15. Springer (2000)
Ebert, T., Nelles, O.: A Note on Analytical Gradient Calculation for Hilomot. Technical report, University of Siegen (2013)
Ebert, T., Fischer, T., Belz, J., Heinz, T.O., Kampmann, G., Nelles, O.: Extended deterministic local search algorithm for maximin Latin hypercube designs. In: 2015 IEEE Symposium Series on Computational Intelligence: IEEE Symposium on Computational Intelligence in Control and Automation (2015 IEEE CICA), Cape Town, South Africa (2015)
Ernst, S.: Hinging hyperplane trees for approximation and identification. In: IEEE Conference on Decision and Control (CDC), pp. 1261–1277, Tampa, USA (1998)
Ernst, S.: Nonlinear system identification with hinging hyperplane trees. In: European Congress on Intelligent Techniques and Soft Computing (EUFIT), pp. 659–663, Aachen, Germany (1998)
Fink, A., Fischer, M., Nelles, O.: Supervison of nonlinear adaptive controllers based on fuzzy models. In: IFAC World Congress, vol. Q, pp. 335–340, Beijing, China (1999)
Fischer, M., Nelles, O., Fink, A.: Adaptive fuzzy model-based control. Journal A 39(3), 22–28 (1998)
Fischer, M., Nelles, O., Isermann, R.: Exploiting prior knowledge in fuzzy model identification of a heat exchanger. In: IFAC Symposium on Artificial Intelligence in Real-Time Control (AIRTC), pp. 445–450, Kuala Lumpur, Malaysia (1997)
Fischer, T., Hartmann, B., Nelles, O.: Increasing the performance of a training algorithm for local model networks. In: World Congress of Engineering and Computer Science (WCECS), pp. 1104–1109, San Francisco, USA (2012)
Fortescue, T.R., Kershenbaum, L.S., Ydstie, B.E.: Implementation of self-tuning regulators with variable forgetting factor. Automatica 17, 831–835 (1981)
Fritzke, B.: Fast learning with incremental radial basis function networks. Neural Process. Lett. 1(1), 2–5 (1994)
Fritzke, B.: Growing cell structures: a self-organizing network for unsupervised and supervised learning. Neural Netw. 7(9), 1441–1460 (1994)
Garulli, A., Paoletti, S., Vicino, A.: A survey on switched and piecewise affine system identification. IFAC Proc. 45(16), 344–355 (2012)
Golub, G., Pereyra, V.: Separable nonlinear least squares: the variable projection method and its applications. Inverse Prob. 19(2), R1 (2003)
Golub, G.H., Pereyra, V.: The differentiation of pseudo-inverses and nonlinear least squares problems whose variables separate. SIAM J. Numer. Anal. 10(2), 413–432 (1973)
Haber, R.: Nonlinearity tests for dynamic processes. In: IFAC Symposium on Identification and System Parameter Estimation, pp. 409–414, York, UK (1985)
Hafner, M., Schüler, M., Isermann, R.: Fast neural networks for diesel engine control design. In: IFAC World Congress, Beijing, China (to appear in 1999)
Hafner, M., Schüler, M., Nelles, O.: Dynamical identification and control of combustion engine exhaust. In: American Control Conference (ACC), pp. 222–226, San Diego, USA (1999)
Hafner, M., Schüler, M., Nelles, O.: Neural net models for Diesel engines – simulation and exhaust optimization. In: European Congress on Intelligent Techniques and Soft Computing (EUFIT), vol. 1, pp. 215–219, Aachen, Germany (1998)
Halfmann, C., Nelles, O., Holzmann, H.: Semi-physical modeling of the vertical vehicle dynamics. In: American Control Conference (ACC), pp. 1707–1711, San Diego, USA (1999)
Hametner, C., Jakubek, S.: Neuro-fuzzy modelling using a logistic discriminant tree. In: 2007 American Control Conference, pp. 864–869. IEEE (2007)
Hametner, C., Jakubek, S.: Nonlinear system identification through local model approaches: partitioning strategies and parameter estimation. INTECH Open Access Publisher (2010)
Hametner, C., Jakubek, S.: Local model network identification for online engine modelling. Inf. Sci. 220, 210–225 (2013)
Hametner, C., Stadlbauer, M., Deregnaucourt, M., Jakubek, S., Winsel, T.: Optimal experiment design based on local model networks and multilayer perceptron networks. Eng. Appl. Artif. Intell. 26(1), 251–261 (2013)
Hartmann, B., Ebert, T., Nelles, O.: Model-based design of experiments based on local model networks for nonlinear processes with low noise levels. In: American Control Conference (ACC), pp. 5306–5311, 29 2011–July 1 2011 (2011)
Hartmann, B., Moll, J., Nelles, O., Fritzen, C.-P.: Hierarchical local model trees for design of experiments in the framework of ultrasonic structural health monitoring. In: IEEE International Conference on Control Applications (CCA), pp. 1163–1170. IEEE (2011)
Hartmann, B., Nelles, O.: Automatic adjustment of the transition between local models in a hierarchical structure identification algorithm. In: European Control Conference (ECC), Budapest, Hungary (2009)
Hartmann, B., Nelles, O.: Adaptive test planning for the calibration of combustion engines – methodology. In: Design of Experiments (DoE) in Engine Development, pp. 1–16, Berlin, Germany. Expert Verlag (2013)
Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning. Springer Series in Statistics, 2nd edn. Springer, Berlin (2009)
Hathaway, R.J., Bezdek, J.C.: Switching regression models and fuzzy clustering. IEEE Trans. Fuzzy Syst. 1(3), 195–204 (1993)
Heinz, T.O., Belz, J., Nelles, O.: Design of experiments – combining linear and nonlinear inputs. In: Hoffman, F., Hüllermeier, E., Mikut, R. (eds.) Proceedings 27. Workshop Computational Intelligence, pp. 211–226. KIT Scientific Publishing (2017)
Hu, C., Wan, F.: Input selection in learning systems: a brief review of some important issues and recent developments. In: IEEE International Conference on Fuzzy Systems. FUZZ-IEEE 2009, pp. 530–535. IEEE (2009)
Isermann, R.: Identifikation dynamischer Syteme – Band 1, 2. ed. Springer, Berlin (1992)
Jakubek, S., Keuth, N.: A new training algorithm for neuro-fuzzy networks. In: ANNIIP, pp. 23–34 (2005)
Jang, J.-S.R.: Neuro-Fuzzy Modeling: Architectures, Analyses, and Applications. Ph.D. thesis, EECS Department, Univ. of California at Berkeley, Berkeley, USA (1992)
Jang, J.-S.R.: ANFIS: adaptive-network-based fuzzy inference systems. IEEE Trans. Syst. Man Cybern. 23(3), 665–685 (1993)
Jang, J.-S.R., Sun, C.T., Mizutani, E.: Neuro-Fuzzy and Soft Computing: A Computational Approach to Learning and Machine Intelligence. Prentice Hall, Englewood Cliffs (1997)
Johansen, T.A., Foss, B.A.: Constructing NARMAX models using ARMAX models. Int. J. Control 58(5), 1125–1153 (1993)
Junge, T.F., Unbehauen, H.: Real time learning control of an emergency turbo-generator plant using structurally adaptive neural networks. In: IEEE Conference on Industrial Electronics (IECON), pp. 2403–2408, Aachen, Germany (1998)
Kadirkamanathan, V., Fabri, S.: Stable nonlinear adaptive control with growing radial basis function networks. In: IFAC Symposium on Adaptive Systems for Control and Signal Processing (ASCAP), pp. 231–236, Budapest, Hungary (1995)
Klein, P., Kirschbaum, F., Hartmann, B., Bogachik, J., Nelles, O.: Adaptive test planning for the calibration of combustion engines – application. In: Design of Experiments (DoE) in Engine Development, pp. 17–30, Berlin, Germany. Expert Verlag (2013)
Kofahl, R.: Robuste parameteradaptive Regelungen. Fachbericht Nr. 19. Messen, Steuern, Regeln. Springer, Berlin (1988)
Kramer, M.A.: Diagnosing dynamic faults using modular neural nets. IEEE Expert (1993)
Kroll, A.: Fuzzy-Systeme zur Modellierung und Regelung komplexer technischer Systeme. Reihe 8: Mess-, Steuerungs- und Regelungstechnik, Nr. 612. VDI-Verlag, Düsseldorf (1997)
Leonhard, J.A., Kramer, M.A., Ungar, L.H.: A neural network architecture that computes its own reliability. Comput. Chem. Eng. 16(9), 818–835 (1992)
Ljung, L.: System Identification: Theory for the User, 2nd edn. Prentice Hall, Englewood Cliffs (1999)
Lughofer, E.: Evolving Fuzzy Systems-Methodologies, Advanced Concepts and Applications, vol. 53. Springer (2011)
Moll, J., Schulte, R.T., Hartmann, B., Fritzen, C.-P., Nelles, O.: Multi-site damage localization in anisotropic plate-like structures using an active guided wave structural health monitoring system. Smart Mater. Struct. 19(4), 045022 (2010)
Müller, N., Nelles, O.: Closed-loop ignition control using on-line learning of locally-tuned radial basis function networks. In: American Control Conference (ACC), pp. 1356–1360, San Diego, USA (1999)
Murray-Smith, R.: A Local Model Network Approach to Nonlinear Modeling. Ph.D. thesis, University of Strathclyde, Strathclyde, UK (1994)
Nakamori, Y., Ryoke, M.: Identification of fuzzy prediction models through hyperellipsoidal clustering. IEEE Trans. Syst. Man Cybern 24(8), 1153–1173 (1994)
Nelles, O.: Local linear model tree for on-line identification of time-variant nonlinear dynamic systems. In: International Conference on Artificial Neural Networks (ICANN), pp. 115–120, Bochum, Germany (1996)
Nelles, O.: LOLIMOT – Lokale, lineare Modelle zur Identifikation nichtlinearer, dynamischer Systeme. Automatisierungstechnik (at) 45(4), 163–174 (1997)
Nelles, O.: Structure optimization of Takagi-Sugeno fuzzy models. Int. J. Uncertainty Fuzziness Knowledge Based Syst. 5(2), 161–170 (1998). Special Issue on Applications of New Functional Principles of Fuzzy Systems and Neural Networks within Computational Intelligence
Nelles, O.: Axes-oblique partitioning strategies for local model networks. In: IEEE International Symposium on Intelligent Control, pp. 2378–2383, Munich, Germany (2006)
Nelles, O., Fischer, M.: Lokale Linearisierung von Fuzzy-Modellen. Automatisierungstechnik (at) 47(5), 217–223 (1999)
Nelles, O., Hecker, O., Isermann, R.: Automatic model selection in local linear model trees for nonlinear system identification of a transport delay process. In: IFAC Symposium on System Identification (SYSID), pp. 727–732, Kitakyushu, Fukuoka, Japan (1997)
Nelles, O., Hecker, O., Isermann, R.: Identifikation nichtlinearer, dynamischer Prozesse mit Takagi-Sugeno Fuzzy-Modellen variabler Struktur. In: 4. GI Fuzzy-Neuro-Systeme Workshop, pp. 388–395, Soest, Germany (1997)
Nelles, O., Hecker, O., Isermann, R.: Automatische Strukturselektion für Fuzzy-Modelle zur Identifikation nichtlinearer, dynamischer Prozesse. Automatisierungstechnik (at) 46(6), 302–312 (1998)
Nelles, O., Isermann, R.: Basis function networks for interpolation of local linear models. In: IEEE Conference on Decision and Control (CDC), pp. 470–475, Kobe, Japan (1996)
Paoletti, S., Juloski, A.L., Ferrari-Trecate, G., Vidal, R.: Identification of hybrid systems a tutorial. Eur. J. Control 13(2), 242–260 (2007)
Park, M.-K., Ji, S.-H., Kim, E.-T., Park, M.: Identification of Takagi-Sugeno fuzzy models via clustering and hough transform. In: Hellendoorn, H., Driankov, D. (eds.) Fuzzy Model Identification: Selected Approaches, chapter 3, pp. 91–161. Springer, Berlin (1997)
Plutowski, M.: Selecting Training Examplars for Neural Network Learning. Ph.D. thesis, University of California, San Diego, USA (1994)
Pucar, P., Millnert, M.: Smooth hinging hyperplanes: a alternative to neural nets. In: European Control Conference (ECC), pp. 1173–1178, Rome, Italy (1995)
Ronco, E., Gawthrop, P.J.: Incremental model reference adaptive polynomial controller network. In: IEEE Conference on Decision and Control, pp. 4171–4172, New York, USA (1997)
Runkler, T.A., Bezdek, J.C.: Polynomial membership functions for smooth first order Takagi-Sugeno systems. In: GI-Workshop Fuzzy-Neuro-Systeme: Computaional Intelligence, pp. 382–387, Soest, Germany (1997)
Sanner, R.M., Slotine, J.-J.E.: Gaussian networks for direct adaptive control. IEEE Trans. Neural Netw. 3, 837–863 (1992)
Schmidt, M., Nelles, O.: Filtering and deriving signals using neural networks. In: American Control Conference (ACC), pp. 2730–2731, Philadelphia, USA (1998)
Schwarz, R.: Rekonstruktion der Bremskraft bei Fahrzeugen mit elektromechanisch betätigten Radbremsen. Ph.D. Thesis, TU Darmstadt, Reihe 12: Verkehrstechnik/Fahrzeugtechnik, Nr. 393. VDI-Verlag, Düsseldorf (1999)
Schwarz, R., Nelles, O., Isermann, R.: Verbesserung der Signalgenauigkeit von Raddrehzahlsensoren mittels Online-Kompensation der Impulsradfehler. Automatisierungstechnische Praxis (atp) 41(3), 35–42 (1999)
Schwarz, R., Nelles, O., Isermann, R., Scheerer, P.: Verfahren zum Ausgleich von Abweichungen eines Raddrehzahlsensors. Patent DE 197 21 488 A1 (1999)
Schwarz, R., Nelles, O., Scheerer, P., Isermann, R.: Increasing signal accuracy of automatic wheel-speed sensors by on-line learning. In: American Control Conference (ACC), pp. 1131–1135, Albuquerque, USA (1997)
Settles, B.: Active learning literature survey. Univ. Wisconsin Madison 52(55–66), 11 (2010)
Thrun, S.B.: The role of exploration in learning control. In: Handbook of Intelligent Control: Neural Fuzzy and Adaptive Approaches. Van Nostrand Reinhold (1992)
Tibshirani, R.J.: Degrees of freedom and model search. arXiv preprint arXiv:1402.1920 (2014)
Tulleken, H.J.A.F.: Grey-box modelling and identification using prior knowledge and Bayesian techniques. Automatica 29(2), 285–308 (1993)
Yoshinari, Y., Pedrycz, W., Hirota, K.: Construction of fuzzy models through clustering techniques. Fuzzy Sets Syst. 54, 157–165 (1993)
Zhou, Z.-H., Wu, J., Tang, W.: Ensembling neural networks: many could be better than all. Artif. Intell. 137(1), 239–263 (2002)
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2020 The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Nelles, O. (2020). Local Linear Neuro-Fuzzy Models: Advanced Aspects. In: Nonlinear System Identification. Springer, Cham. https://doi.org/10.1007/978-3-030-47439-3_14
Download citation
DOI: https://doi.org/10.1007/978-3-030-47439-3_14
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-47438-6
Online ISBN: 978-3-030-47439-3
eBook Packages: Physics and AstronomyPhysics and Astronomy (R0)