Skip to main content
SpringerLink
Log in

Fuzzy evolving linear regression trees

  • Original Paper
  • Published:
Evolving Systems Aims and scope Submit manuscript

Abstract

This paper introduces a new approach for evolving fuzzy modeling using tree structures. The model is a fuzzy linear regression tree whose topology can be continuously updated through a statistical model selection test. A fuzzy linear regression tree is a fuzzy tree with linear model in each leaf. An incremental learning algorithm approach evolves the tree replacing leaves with subtrees that improve the model quality. The learning algorithm evaluates each input only once and do not need to store any past values. The evolving linear regression model is evaluated using time series forecasting problems. The performance is compared against alternative evolving fuzzy models and classic models with fixed structures. The results suggest that fuzzy evolving regression tree is a promising approach for adaptive system modeling.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  • Allen MP (1997) Understanding regression analysis, 1st edn. Springer, New York

  • Angelov P, Filev D (2004) An approach to online identification of Takagi–Sugeno fuzzy models. IEEE Trans Syst Man Cybern B: Cybern 34(1):484–498

    Article  Google Scholar 

  • Angelov P, Filev D (2005) Simpl_ets: a simplified method for learning evolving Takagi–Sugeno fuzzy models. In: The 14th IEEE international conference on fuzzy systems, 2005 (FUZZ ’05), May 2005, pp 1068–1073

  • Angelov P, Guthke R (1997) A genetic-algorithm-based approach to optimization of bioprocesses described by fuzzy rules. Bioprocess Eng 16(5):299–303

    Article  Google Scholar 

  • Angelov P, Zhou X (2006) Evolving fuzzy systems from data streams in real-time. In: 2006 International symposium on evolving fuzzy systems, September 2006, pp 29–35

  • Angelov P, Filev D, Kasabov N (2008) Guest editorial evolving fuzzy systems—preface to the special section. IEEE Trans Fuzzy Syst 16(6):1390–1392

    Article  Google Scholar 

  • Babuska R (1998) Fuzzy modeling for control. Kluwer, Boston

    Google Scholar 

  • Bezdek JC (1981) Pattern recognition with fuzzy objective function algorithms. Kluwer, Norwell. ISBN:0306406713

  • Box GEP, Jenkins G (1990) Time series analysis, forecasting and control. Holden-Day. ISBN:0816211043

  • Breiman J, Friedman L, Olshen R, Stone C (1984) Classification and regression trees. Wadsworth Mathematics Series, San Diego

  • Chiu S (1994) Fuzzy model identification based on cluster estimation. J Intell Fuzzy Syst 2(3):267–278

    Google Scholar 

  • Diebold FX, Mariano RS (1995) Comparing predictive accuracy. J Bus Econ Stat 13:253–263

    Article  Google Scholar 

  • Dobra A, Gehrke J (2002) Secret: a scalable linear regression tree algorithm. In: KDD ’02: Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, New York, NY, USA. ACM, New York, pp 481–487. ISBN:1-58113-567-X. doi:10.1145/775047.775117

  • Duda RO, Hart PE, Stork DG (2000) Pattern classification, 2nd edn. Wiley-Interscience

  • Filev D, Angelov P (2007) Algorithms for real-time clustering and generation of rules from data. In: Oliveira JVD, Pedrycz W (eds) Advances in fuzzy clustering and its applications. Wiley, New York

    Google Scholar 

  • Gray R (1984) Vector quantization. IEEE ASSP Mag 1(2):4–29

    Article  Google Scholar 

  • Hyndman RJ (2010) Time series data library. http://robjhyndman.com/TSDL. Accessed December 2010

  • Ikonomovska E, Gama J (2008) Learning model trees from data streams. In: Boulicaut J-F, Berthold M, Horvth T (eds) Discovery science. Lecture notes in computer science, vol 5255. Springer, Berlin, pp 52–63. doi:10.1007/978-3-540-88411-8_8

  • Ikonomovska E, Gama J, Sebastião R, Gjorgjevik D (2009) Regression trees from data streams with drift detection. In: Proceedings of the 12th international conference on discovery science, DS ’09. Springer, Berlin, pp 121–135. ISBN:978-3-642-04746-6. doi:10.1007/978-3-642-04747-3_12

  • Jang J (1993) ANFIS—adaptive-network-based fuzzy inference system. IEEE Trans Syst Man Cybern 23(3):665–685

    Google Scholar 

  • Jang J-SR, Sun C-T, Mizutani E (1997) Neuro-fuzzy and soft computing: a computational approach to learning and machine intelligence, 1st edn. Prentice-Hall, Englewood Cliffs

  • Janikow C (1998) Fuzzy decision trees: issues and methods. IEEE Trans Syst Man Cybern B: Cyber 28(1):1–14

    Article  Google Scholar 

  • Juang C, Lin C (1998) An on-line self-constructing neural fuzzy inference network and its applications. IEEE Trans Fuzzy Syst 6(1):12–32

    Article  Google Scholar 

  • Kasabov NK (1996) Foundations of neural networks, fuzzy systems, and knowledge engineering. MIT Press, Cambridge

    MATH  Google Scholar 

  • Kasabov N, Filev D (2006) Evolving intelligent systems: methods, learning, & applications. In: 2006 International symposium on evolving fuzzy systems, September 2006, pp 8–18

  • Kasabov N, Song Q (2002) DENFIS: dynamic evolving neural-fuzzy inference system and its application for time-series prediction. IEEE Trans Fuzzy Syst 10(2):144–154

    Article  Google Scholar 

  • Ljung L (ed) (1999) System identification: theory for the user, 2nd edn. Prentice Hall PTR, Upper Saddle River

  • Lughofer ED (2008a) FLEXFIS: a robust incremental learning approach for evolving Takagi–Sugeno fuzzy models. IEEE Trans Fuzzy Syst 16(6):1393–1410

    Article  Google Scholar 

  • Lughofer ED (2008b) Extensions of vector quantization for incremental clustering. Pattern Recognit 41(3):995–1011

    MATH  Google Scholar 

  • Mamdani E, Assilian S (1975) Experiment in linguistic synthesis with a fuzzy logic controller. Int J Man-Mach Stud 7(1):1–13

    Article  MATH  Google Scholar 

  • Miller R (1966) Simultaneous statistical inference. McGraw-Hill, New York

    MATH  Google Scholar 

  • Montgomery D (2011) Introduction to statistical quality control, 4th edn. Wiley, New York

  • Nelles O (2001) Nonlinear system identification. Springer, Berlin

    MATH  Google Scholar 

  • Pedrycz W, Gomide F (2007) Fuzzy systems engineering: toward human-centric computing. Wiley-Interscience, New Jersey

    Google Scholar 

  • Potts D (2004) Incremental learning of linear model trees. In: ICML ’04: Proceedings of the twenty-first international conference on machine learning. ACM, New York, p 84. ISBN:1-58113-828-5. doi:10.1145/1015330.1015372

  • Pouzols F, Lendasse A (2010) Evolving fuzzy optimally pruned extreme learning machine for regression problems. Evol Syst 1:43–58. ISSN:1868-6478. doi:10.1007/s12530-010-9005-y

    Google Scholar 

  • Quinlan R (1992) Learning with continuous classes. In: 5th Australian joint conference on artificial intelligence, pp 236–243

  • Rahman S, Hazim O (1993) A generalized knowledge-based short-term load-forecasting technique. IEEE Trans Power Syst 8(2):508–514

    Article  Google Scholar 

  • Rubio J (2009) SOFMLS: online self-organizing fuzzy modified least-squares network. IEEE Trans Fuzzy Syst 17(6):1296–1309

    Article  Google Scholar 

  • Torgo L (1997) Functional models for regression tree leaves. In: ICML ’97: Proceedings of the fourteenth international conference on machine learning, San Francisco, CA, USA. Morgan Kaufmann, pp 385–393. ISBN:1-55860-486-3

  • Wang L, Mendel J (1992) Generating fuzzy rules by learning from examples. IEEE Trans Syst Man Cybern 22(6):1414–1427

    Article  MathSciNet  Google Scholar 

  • Wang D, Zeng X, Keane J (2010a) A structure evolved learning method for mamdani fuzzy systems. In: AISB-EIS 10, pp 1068–1073

  • Wang D, Zeng X-J, Keane J (2010b) A structure evolving learning method for fuzzy systems. Evol Syst 1:83–95

    Article  Google Scholar 

  • Yager R, Filev D (1994) Approximate clustering via the mountain method. IEEE Trans Syst Man Cybern 24(8):1279–1284

    Article  Google Scholar 

  • Young P (1984) Recursive estimation and time-series analysis: an introduction. Springer, New York. ISBN:0-387-13677-0

  • Yuan Y, Shaw M (1995) Induction of fuzzy decision trees. Fuzzy Sets Syst 69(2):125–139

    Google Scholar 

Download references

Acknowledgments

The authors thank the Brazilian National Research Council, CNPq, for grants 141323/2009-4, 309666/2007-4 and 304596/2009-4, respectively. The second author also acknowledges FAPEMIG, the Research Foundation of the State of Minas Gerais, for grant PPM-00252-09. The help, suggestions, and comments of the anonymous referees are also kindly acknowledged.

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, Federal University of Minas Gerais, Belo Horizonte, 31270-010, Brazil

    Andre Lemos

  2. Department of Electronic Engineering, Federal University of Minas Gerais, Belo Horizonte, 31270-010, Brazil

    Walmir Caminhas

  3. School of Electrical and Computer Engineering, University of Campinas, Campinas, 13083-970, Brazil

    Fernando Gomide

Authors
  1. Andre Lemos
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Walmir Caminhas
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fernando Gomide
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Andre Lemos.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Lemos, A., Caminhas, W. & Gomide, F. Fuzzy evolving linear regression trees. Evolving Systems 2, 1–14 (2011). https://doi.org/10.1007/s12530-011-9028-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12530-011-9028-z

Keywords

Search

Navigation