Skip to main content
Erschienen in: Soft Computing 8/2014

01.08.2014 | Foundations

Robust on-line neural learning classifier system for data stream classification tasks

verfasst von: Andreu Sancho-Asensio, Albert Orriols-Puig, Elisabet Golobardes

Erschienen in: Soft Computing | Ausgabe 8/2014

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

The increasing integration of technology in the different areas of science and industry has resulted in the design of applications that generate large amounts of data on-line. Most often, extracting information from these data is key, in order to gain a better understanding of the processes that the data are describing. Learning from these data poses new challenges to traditional machine learning techniques, which are not typically designed to deal with data in which concepts and noise levels may vary over time. The purpose of this paper is to present supervised neural constructivist system (SNCS), an accuracy-based neural-constructivist learning classifier system that makes use of multilayer perceptrons to learn from data streams with a fast reaction capacity to concept changes. The behavior of SNCS on data stream problems with different characteristics is carefully analyzed and compared with other state-of-the-art techniques in the field. This comparison is also extended to a large collection of real-world problems. The results obtained show that SNCS can function in a variety of problem situations producing accurate classification of data, whether the data are static or in dynamic streams.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Anhänge
Nur mit Berechtigung zugänglich
Fußnoten
1
Discriminatory functions allow to tell whether a measure is zero or not.
 
Literatur
Zurück zum Zitat Aggarwal CC (ed) (2007) Data streams—models and algorithms, advances in database systems, vol 31. Springer, Berlin Aggarwal CC (ed) (2007) Data streams—models and algorithms, advances in database systems, vol 31. Springer, Berlin
Zurück zum Zitat Aha DW, Kibler D, Albert MK (1991) Instance-based learning algorithms. Mach Learn 6(1):37–66 Aha DW, Kibler D, Albert MK (1991) Instance-based learning algorithms. Mach Learn 6(1):37–66
Zurück zum Zitat Alcalá-Fdez J, Sánchez L, García S, del Jesus M, Ventura S, Garrell J, Otero J, Romero C, Bacardit J, Rivas V, Fernández J, Herrera F (2009) Keel: a software tool to assess evolutionary algorithms for data mining problems. Soft Comput Fusion Found Methodol Appl 13:307–318 Alcalá-Fdez J, Sánchez L, García S, del Jesus M, Ventura S, Garrell J, Otero J, Romero C, Bacardit J, Rivas V, Fernández J, Herrera F (2009) Keel: a software tool to assess evolutionary algorithms for data mining problems. Soft Comput Fusion Found Methodol Appl 13:307–318
Zurück zum Zitat Angelov P, Zhou X (2008) Evolving fuzzy-rule-based classifiers from data streams. IEEE Trans Fuzzy Syst 16(6):1462–1475CrossRef Angelov P, Zhou X (2008) Evolving fuzzy-rule-based classifiers from data streams. IEEE Trans Fuzzy Syst 16(6):1462–1475CrossRef
Zurück zum Zitat Baruah R, Angelov P (2012) Evolving local means method for clustering of streaming data. In: 2012 IEEE international conference on Fuzzy systems (FUZZ-IEEE), pp 1–8 Baruah R, Angelov P (2012) Evolving local means method for clustering of streaming data. In: 2012 IEEE international conference on Fuzzy systems (FUZZ-IEEE), pp 1–8
Zurück zum Zitat Bull L (2002b) On using constructivism in neural classifier systems. In: Proceedings of the 7th international conference on parallel problem solving from nature. Springer, London, PPSN VII, pp 558–567 Bull L (2002b) On using constructivism in neural classifier systems. In: Proceedings of the 7th international conference on parallel problem solving from nature. Springer, London, PPSN VII, pp 558–567
Zurück zum Zitat Bull L, O’Hara T (2002) Accuracy-based neuro and neuro-fuzzy classifier systems. In: GECCO ’02: Proceedings of the genetic and evolutionary computation Conference. Morgan Kaufmann, New York, USA, pp 905–911 Bull L, O’Hara T (2002) Accuracy-based neuro and neuro-fuzzy classifier systems. In: GECCO ’02: Proceedings of the genetic and evolutionary computation Conference. Morgan Kaufmann, New York, USA, pp 905–911
Zurück zum Zitat Butz MV (2006) Rule-based evolutionary online learning systems—a principled approach to LCS analysis and design, studies in fuzziness and soft computing, vol 191. Springer, Berlin Butz MV (2006) Rule-based evolutionary online learning systems—a principled approach to LCS analysis and design, studies in fuzziness and soft computing, vol 191. Springer, Berlin
Zurück zum Zitat Butz MV, Sastry K, Goldberg DE (2005) Strong, stable, and reliable fitness pressure in XCS due to tournament selection. Genet Program Evolvable Mach 6(1):53–77CrossRef Butz MV, Sastry K, Goldberg DE (2005) Strong, stable, and reliable fitness pressure in XCS due to tournament selection. Genet Program Evolvable Mach 6(1):53–77CrossRef
Zurück zum Zitat Chandra B, Bhaskar S (2011) A novel approach of finding frequent itemsets in high speed data streams. In: Eighth international conference on fuzzy systems and knowledge discovery, FSKD, vol 1, pp 40–44 Chandra B, Bhaskar S (2011) A novel approach of finding frequent itemsets in high speed data streams. In: Eighth international conference on fuzzy systems and knowledge discovery, FSKD, vol 1, pp 40–44
Zurück zum Zitat Demšar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30MATHMathSciNet Demšar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30MATHMathSciNet
Zurück zum Zitat Derrac J, García S, Molina D, Herrera F (2011) A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms. Swarm Evol Comput 1(1):3–18CrossRef Derrac J, García S, Molina D, Herrera F (2011) A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms. Swarm Evol Comput 1(1):3–18CrossRef
Zurück zum Zitat Dietterich TG (1998) Approximate statistical tests for comparing supervised classification learning algorithms. Neural Comput 10:1895–1923CrossRef Dietterich TG (1998) Approximate statistical tests for comparing supervised classification learning algorithms. Neural Comput 10:1895–1923CrossRef
Zurück zum Zitat Edwin L, Angelov PP (2011) Handling drifts and shifts in on-line data streams with evolving fuzzy systems. Appl Soft Comput 11(2):2057–2068CrossRef Edwin L, Angelov PP (2011) Handling drifts and shifts in on-line data streams with evolving fuzzy systems. Appl Soft Comput 11(2):2057–2068CrossRef
Zurück zum Zitat Fan W (2004) Systematic data selection to mine concept-drifting data streams. In: Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, ACM, New York, USA, KDD ’04, pp 128–137 Fan W (2004) Systematic data selection to mine concept-drifting data streams. In: Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, ACM, New York, USA, KDD ’04, pp 128–137
Zurück zum Zitat Friedman M (1937) The use of ranks to avoid the assumption of normality implicit in the analysis of variance. J Am Stat Assoc 32(200):675–701CrossRef Friedman M (1937) The use of ranks to avoid the assumption of normality implicit in the analysis of variance. J Am Stat Assoc 32(200):675–701CrossRef
Zurück zum Zitat Gama J (ed) (2010) Knowledge discovery from data streams. Chapman and Hall/CRC data mining and knowledge discovery series, 1st edn. CRC Press. pp I–XIX, 1–237 Gama J (ed) (2010) Knowledge discovery from data streams. Chapman and Hall/CRC data mining and knowledge discovery series, 1st edn. CRC Press. pp I–XIX, 1–237
Zurück zum Zitat Gama J, Gaber MM (eds) (2007) Learning from data streams: processing techniques in sensor networks, 1st edn. Springer, Berlin Gama J, Gaber MM (eds) (2007) Learning from data streams: processing techniques in sensor networks, 1st edn. Springer, Berlin
Zurück zum Zitat García S, Herrera F (2008) An extension on “Statistical comparisons of classifiers over multiple data sets” for all pairwise comparisons. J Mach Learn Res 9:2677–2694MATH García S, Herrera F (2008) An extension on “Statistical comparisons of classifiers over multiple data sets” for all pairwise comparisons. J Mach Learn Res 9:2677–2694MATH
Zurück zum Zitat Goldberg DE (2002) The design of innovation: lessons from and for competent genetic algorithms. Kluwer Academic Publishers, NorwellCrossRef Goldberg DE (2002) The design of innovation: lessons from and for competent genetic algorithms. Kluwer Academic Publishers, NorwellCrossRef
Zurück zum Zitat Holland J (1992) Adaptation in natural and artificial systems, 2nd edn. The MIT Press, Cambridge Holland J (1992) Adaptation in natural and artificial systems, 2nd edn. The MIT Press, Cambridge
Zurück zum Zitat Holm S (1979) A simple sequentially rejective multiple test procedure. Scand J Stat 6:65–70MATHMathSciNet Holm S (1979) A simple sequentially rejective multiple test procedure. Scand J Stat 6:65–70MATHMathSciNet
Zurück zum Zitat Howard G, Bull L, Lanzi P (2009) Towards continuous actions in continuous space and time using self-adaptive constructivism in neural XCSF. In: GECCO ’09: Proceedings of the 11th annual conference on genetic and evolutionary computation. ACM, New York, USA, pp 1219–1226 Howard G, Bull L, Lanzi P (2009) Towards continuous actions in continuous space and time using self-adaptive constructivism in neural XCSF. In: GECCO ’09: Proceedings of the 11th annual conference on genetic and evolutionary computation. ACM, New York, USA, pp 1219–1226
Zurück zum Zitat Howard G, Bull L, Lanzi PL (2010) A spiking neural representation for xcsf. In: 2010 IEEE congress on evolutionary computation (CEC), pp 1–8 Howard G, Bull L, Lanzi PL (2010) A spiking neural representation for xcsf. In: 2010 IEEE congress on evolutionary computation (CEC), pp 1–8
Zurück zum Zitat Hulten G, Spencer L, Domingos P (2001) Mining time-changing data streams. In: Proceedings of the 2001 ACM SIGKDD international conference on knowledge discovery and data mining, pp 97–106 Hulten G, Spencer L, Domingos P (2001) Mining time-changing data streams. In: Proceedings of the 2001 ACM SIGKDD international conference on knowledge discovery and data mining, pp 97–106
Zurück zum Zitat John G, Langley P (1995) Estimating continuous distributions in bayesian classifiers. In: Proceedings of the eleventh conference on uncertainty in artificial intelligence, Morgan Kaufmann, pp 338–345 John G, Langley P (1995) Estimating continuous distributions in bayesian classifiers. In: Proceedings of the eleventh conference on uncertainty in artificial intelligence, Morgan Kaufmann, pp 338–345
Zurück zum Zitat Khan L (2010) Data stream mining: challenges and techniques. In: 22nd IEEE international conference on tools with artificial intelligence, ICTAI, vol 2, p 295 Khan L (2010) Data stream mining: challenges and techniques. In: 22nd IEEE international conference on tools with artificial intelligence, ICTAI, vol 2, p 295
Zurück zum Zitat Koščak J, Jakša R, Sinčák P (2010) Stochastic weight update in the backpropagation algorithm on feed-forward neural networks. In: The 2010 international joint conference on neural networks (IJCNN), pp 1–4 Koščak J, Jakša R, Sinčák P (2010) Stochastic weight update in the backpropagation algorithm on feed-forward neural networks. In: The 2010 international joint conference on neural networks (IJCNN), pp 1–4
Zurück zum Zitat Lakshmi K, Reddy C (2010) A survey on different trends in data streams. In: 2010 International conference on networking and information technology (ICNIT), pp 451–455 Lakshmi K, Reddy C (2010) A survey on different trends in data streams. In: 2010 International conference on networking and information technology (ICNIT), pp 451–455
Zurück zum Zitat Masud MM, Gao J, Khan L, Han J, Thuraisingham BM (2011) Classification and novel class detection in concept-drifting data streams under time constraints. IEEE Trans Knowl Data Eng 23(6):859–874CrossRef Masud MM, Gao J, Khan L, Han J, Thuraisingham BM (2011) Classification and novel class detection in concept-drifting data streams under time constraints. IEEE Trans Knowl Data Eng 23(6):859–874CrossRef
Zurück zum Zitat Mitchell T (1997) Machine learning. Prentice Hall, PittsburghMATH Mitchell T (1997) Machine learning. Prentice Hall, PittsburghMATH
Zurück zum Zitat Mozafari N, Hashemi S, Hamzeh A (2011) A precise statistical approach for concept change detection in unlabeled data streams. Comput Math Appl 62(4):1655–1669CrossRefMATHMathSciNet Mozafari N, Hashemi S, Hamzeh A (2011) A precise statistical approach for concept change detection in unlabeled data streams. Comput Math Appl 62(4):1655–1669CrossRefMATHMathSciNet
Zurück zum Zitat Nemenyi P (1964) Distribution-free multiple comparisons. PhD thesis, Princeton University, New Jersey, USA Nemenyi P (1964) Distribution-free multiple comparisons. PhD thesis, Princeton University, New Jersey, USA
Zurück zum Zitat Núñez M, Fidalgo R, Morales R (2007) Learning in environments with unknown dynamics: towards more robust concept learners. J Mach Learn Res 8:2595–2628MATHMathSciNet Núñez M, Fidalgo R, Morales R (2007) Learning in environments with unknown dynamics: towards more robust concept learners. J Mach Learn Res 8:2595–2628MATHMathSciNet
Zurück zum Zitat Orriols-Puig A (2008) New challenges in learning classifier systems: mining rarities and evolving fuzzy models (E. Bernadó-Mansilla). PhD thesis, Arquitectura i Enginyeria La Salle, Universitat Ramon Llull, Passeig de la Bonanova 8, 08022, Barcelona Orriols-Puig A (2008) New challenges in learning classifier systems: mining rarities and evolving fuzzy models (E. Bernadó-Mansilla). PhD thesis, Arquitectura i Enginyeria La Salle, Universitat Ramon Llull, Passeig de la Bonanova 8, 08022, Barcelona
Zurück zum Zitat Orriols-Puig A, Casillas J (2011) Fuzzy knowledge representation study for incremental learning in data streams and classification problems. Soft Comput 15(12): 2389–2414 Orriols-Puig A, Casillas J (2011) Fuzzy knowledge representation study for incremental learning in data streams and classification problems. Soft Comput 15(12): 2389–2414
Zurück zum Zitat Platt J (1998) Fast training of support vector machines using sequential minimal optimization. In: Schoelkopf B, Burges C, Smola A (eds) Advances in Kernel methods—support vector learning. MIT Press, Cambridge Platt J (1998) Fast training of support vector machines using sequential minimal optimization. In: Schoelkopf B, Burges C, Smola A (eds) Advances in Kernel methods—support vector learning. MIT Press, Cambridge
Zurück zum Zitat Quartz SR, Sejnowski TJ (1999) The neural basis of cognitive development: a constructivist manifesto Quartz SR, Sejnowski TJ (1999) The neural basis of cognitive development: a constructivist manifesto
Zurück zum Zitat Quinlan JR (1993) C4.5: programs for machine learning. Morgan Kaufmann Publishers, San Francisco Quinlan JR (1993) C4.5: programs for machine learning. Morgan Kaufmann Publishers, San Francisco
Zurück zum Zitat Rumelhart DE, Hinton GE, McClelland JL (eds) (1986) Parallel distributed processing: explorations in the microstructure of cognition, foundations, vol 1. MIT Press, Cambridge Rumelhart DE, Hinton GE, McClelland JL (eds) (1986) Parallel distributed processing: explorations in the microstructure of cognition, foundations, vol 1. MIT Press, Cambridge
Zurück zum Zitat Shaffer JP (1986) Modified sequentially rejective multiple test procedures. J Am Stat Assoc 81:826–831CrossRefMATH Shaffer JP (1986) Modified sequentially rejective multiple test procedures. J Am Stat Assoc 81:826–831CrossRefMATH
Zurück zum Zitat Street WN, Kim Y (2001) A streaming ensemble algorithm (SEA) for large-scale classification. In: KDD ’01: Proceedings of the seventh ACM SIGKDD international conference on knowledge discovery and data mining. ACM, New York, USA, pp 377–382 Street WN, Kim Y (2001) A streaming ensemble algorithm (SEA) for large-scale classification. In: KDD ’01: Proceedings of the seventh ACM SIGKDD international conference on knowledge discovery and data mining. ACM, New York, USA, pp 377–382
Zurück zum Zitat Torres DM, Aguilar-Ruiz JS, Sarabia YR (2012) An instance based learning model for classification in data streams with concept change. In: 11th Mexican international conference on artificial intelligence, MICAI 2012, San Luis Potos, Mexico, October 27–November 4, 2012, special session proceedings, pp 58–62 Torres DM, Aguilar-Ruiz JS, Sarabia YR (2012) An instance based learning model for classification in data streams with concept change. In: 11th Mexican international conference on artificial intelligence, MICAI 2012, San Luis Potos, Mexico, October 27–November 4, 2012, special session proceedings, pp 58–62
Zurück zum Zitat Vivekanandan P, Nedunchezhian R (2011) Mining data streams with concept drifts using genetic algorithm. Artif Intell Rev 36(3):163–178CrossRef Vivekanandan P, Nedunchezhian R (2011) Mining data streams with concept drifts using genetic algorithm. Artif Intell Rev 36(3):163–178CrossRef
Zurück zum Zitat Widmer G, Kubat M (1996) Learning in the presence of concept drift and hidden contexts. Mach Learn 23(1):69–101 Widmer G, Kubat M (1996) Learning in the presence of concept drift and hidden contexts. Mach Learn 23(1):69–101
Zurück zum Zitat Widrow B, Lehr M (1990) 30 Years of adaptive neural networks: perceptron, madaline, and backpropagation. Proc IEEE 78(9):1415–1442CrossRef Widrow B, Lehr M (1990) 30 Years of adaptive neural networks: perceptron, madaline, and backpropagation. Proc IEEE 78(9):1415–1442CrossRef
Zurück zum Zitat Wilson SW (1994) ZCS: a zeroth level classifier system. Tech. rep., The Rowland Institute for Science, 100 Edwin H. Land Blvd, Cambridge 02142 Wilson SW (1994) ZCS: a zeroth level classifier system. Tech. rep., The Rowland Institute for Science, 100 Edwin H. Land Blvd, Cambridge 02142
Zurück zum Zitat Wilson SW (1995) Classifier fitness based on accuracy. Evol Comput 3:149–175CrossRef Wilson SW (1995) Classifier fitness based on accuracy. Evol Comput 3:149–175CrossRef
Zurück zum Zitat Wilson SW (2001) Classifiers that approximate functions. Nat Comput 1:211–234CrossRef Wilson SW (2001) Classifiers that approximate functions. Nat Comput 1:211–234CrossRef
Zurück zum Zitat Wilson SW (2008) Classifier conditions using gene expression programming. In: Bacardit J, Bernadó-Mansilla E, Butz M, Kovacs T, Llorà? X, Takadama K (eds) Learning classifier systems, lecture notes in computer science, vol 4998. Springer, Berlin, pp 206–217 Wilson SW (2008) Classifier conditions using gene expression programming. In: Bacardit J, Bernadó-Mansilla E, Butz M, Kovacs T, Llorà? X, Takadama K (eds) Learning classifier systems, lecture notes in computer science, vol 4998. Springer, Berlin, pp 206–217
Zurück zum Zitat Witten IH, Frank E, Hall MA (2011) Data mining: practical machine learning tools and techniques. Morgan Kaufmann series in data management systems, 3rd edn. Morgan Kaufmann, San Francisco Witten IH, Frank E, Hall MA (2011) Data mining: practical machine learning tools and techniques. Morgan Kaufmann series in data management systems, 3rd edn. Morgan Kaufmann, San Francisco
Zurück zum Zitat Wu X, Kumar V, Ghosh J, Yang Q, Motoda H, McLachlan G, Ng A, Liu B, Yu P, Zhou ZH, Steinbach M, Hand D, Steinberg D (2007) Top 10 algorithms in data mining. Knowl Inf Syst 14:1–37CrossRef Wu X, Kumar V, Ghosh J, Yang Q, Motoda H, McLachlan G, Ng A, Liu B, Yu P, Zhou ZH, Steinbach M, Hand D, Steinberg D (2007) Top 10 algorithms in data mining. Knowl Inf Syst 14:1–37CrossRef
Zurück zum Zitat Zhang GP (2000) Neural networks for classification: a survey. IEEE Trans Syst Man Cybern Part C (Appl Rev) 30(4):451–462CrossRef Zhang GP (2000) Neural networks for classification: a survey. IEEE Trans Syst Man Cybern Part C (Appl Rev) 30(4):451–462CrossRef
Zurück zum Zitat Zhu X, Zhang P, Lin X, Shi Y (2010) Active learning from stream data using optimal weight classifier ensemble. Syst Man Cybern Part B: IEEE Trans Cybern 40(6):1607–1621CrossRef Zhu X, Zhang P, Lin X, Shi Y (2010) Active learning from stream data using optimal weight classifier ensemble. Syst Man Cybern Part B: IEEE Trans Cybern 40(6):1607–1621CrossRef
Metadaten
Titel
Robust on-line neural learning classifier system for data stream classification tasks
verfasst von
Andreu Sancho-Asensio
Albert Orriols-Puig
Elisabet Golobardes
Publikationsdatum
01.08.2014
Verlag
Springer Berlin Heidelberg
Erschienen in
Soft Computing / Ausgabe 8/2014
Print ISSN: 1432-7643
Elektronische ISSN: 1433-7479
DOI
https://doi.org/10.1007/s00500-014-1233-9

Weitere Artikel der Ausgabe 8/2014

Soft Computing 8/2014 Zur Ausgabe