Skip to main content
Erschienen in: Evolutionary Intelligence 3/2009

01.12.2009 | Short Note

Sequential problems that test generalization in learning classifier systems

verfasst von: Martin V. Butz, Pier Luca Lanzi

Erschienen in: Evolutionary Intelligence | Ausgabe 3/2009

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

We present an approach to build sequential decision making problems which can test the generalization capabilities of classifier systems. The approach can be applied to any sequential problem defined over a binary domain and it generates a new problem with bounded sequential difficulty and bounded generalization difficulty. As an example, we applied the approach to generate two problems with simple sequential structure, huge number of states (more than a million), and many generalizations. These problems are used to compare a classifier system with effective generalization (XCS) and a learner without generalization (Q-learning). The experimental results confirm what was previously found mainly using single-step problems: also in sequential problems with huge state spaces, XCS can generalize effectively by detecting those substructures that are necessary for optimal sequential behavior.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Bagnall AJ, Zatuchna ZV (2005) On the classification of maze problems. In: Bull L, Kovacs T (eds) Foundations of learning classifier systems. Studies in fuzziness and soft computing, vol 183. Springer, Berlin, pp 307–316 Bagnall AJ, Zatuchna ZV (2005) On the classification of maze problems. In: Bull L, Kovacs T (eds) Foundations of learning classifier systems. Studies in fuzziness and soft computing, vol 183. Springer, Berlin, pp 307–316
2.
Zurück zum Zitat Barry AM (2002) The stability of long action chains in xcs. J Soft Comput 6(3–4):183–199MATH Barry AM (2002) The stability of long action chains in xcs. J Soft Comput 6(3–4):183–199MATH
3.
Zurück zum Zitat Booker LB (1989) Triggered rule discovery in classifier systems. In: Schaffer JD (ed) Proceedings of the 3rd international conference on genetic algorithms (ICGA89). George Mason University/Morgan Kaufmann, Fairfax county/Menlo Park, pp 265–274 Booker LB (1989) Triggered rule discovery in classifier systems. In: Schaffer JD (ed) Proceedings of the 3rd international conference on genetic algorithms (ICGA89). George Mason University/Morgan Kaufmann, Fairfax county/Menlo Park, pp 265–274
4.
Zurück zum Zitat Bull L (2004) Applications of learning classifier systems. Studies in fuzziness and soft computing. Springer, Berlin Bull L (2004) Applications of learning classifier systems. Studies in fuzziness and soft computing. Springer, Berlin
5.
Zurück zum Zitat Bull L, Mansilla EB, Holmes JH (2008) Learning classifier systems in data mining. Studies in computational intelligence. Springer, Berlin Bull L, Mansilla EB, Holmes JH (2008) Learning classifier systems in data mining. Studies in computational intelligence. Springer, Berlin
6.
Zurück zum Zitat Butz M, Goldberg DE (2003) Generalized state values in an anticipatory learning classifier system. In: Butz M, Sigaud O, Gérard P (eds) Anticipatory behavior in adaptive learning systems. Lecture notes in computer science, vol 2684. Springer, Berlin, pp 282–301 Butz M, Goldberg DE (2003) Generalized state values in an anticipatory learning classifier system. In: Butz M, Sigaud O, Gérard P (eds) Anticipatory behavior in adaptive learning systems. Lecture notes in computer science, vol 2684. Springer, Berlin, pp 282–301
7.
Zurück zum Zitat Butz MV (2000) Anticipatory learning classifier systems. Genetic algorithms and evolutionary computation, vol 4. Springer, Berlin Butz MV (2000) Anticipatory learning classifier systems. Genetic algorithms and evolutionary computation, vol 4. Springer, Berlin
8.
Zurück zum Zitat Butz MV (2006) Rule-based evolutionary online learning systems: a principled approach to LCS analysis and design. Studies in fuzziness and soft computing. Springer, Berlin Butz MV (2006) Rule-based evolutionary online learning systems: a principled approach to LCS analysis and design. Studies in fuzziness and soft computing. Springer, Berlin
9.
Zurück zum Zitat Butz MV, Goldberg DE, Lanzi PL (2005) Gradient descent methods in learning classifier systems: Improving xcs performance in multistep problems. IEEE Trans Evol Comput 9(5):452–473CrossRef Butz MV, Goldberg DE, Lanzi PL (2005) Gradient descent methods in learning classifier systems: Improving xcs performance in multistep problems. IEEE Trans Evol Comput 9(5):452–473CrossRef
10.
Zurück zum Zitat Butz MV, Kovacs T, Lanzi PL, Wilson SW (2004) Toward a theory of generalization and learning in XCS. IEEE Trans Evol Comput 8(1):28–46CrossRef Butz MV, Kovacs T, Lanzi PL, Wilson SW (2004) Toward a theory of generalization and learning in XCS. IEEE Trans Evol Comput 8(1):28–46CrossRef
11.
Zurück zum Zitat Butz MV, Pelikan M, Llorá X, Goldberg DE (2006) Automated global structure extraction for effective local building block processing in XCS. Evol Comput 14:345–380CrossRef Butz MV, Pelikan M, Llorá X, Goldberg DE (2006) Automated global structure extraction for effective local building block processing in XCS. Evol Comput 14:345–380CrossRef
12.
Zurück zum Zitat Butz MV, Wilson SW (2002) An algorithmic description of XCS. J Soft Comput 6(3–4):144–153MATH Butz MV, Wilson SW (2002) An algorithmic description of XCS. J Soft Comput 6(3–4):144–153MATH
14.
Zurück zum Zitat Drugowitsch J, Barry AM (2005) XCS with eligibility traces. In: Beyer AM, O’Reilly U-M, Arnold DV, Banzhaf W, Blum C, Bonabeau EW, Cantu-Paz E, Dasgupta D, Deb K, Foster JA, de Jong ED, Lipson H, Llora X, Mancoridis S, Pelikan M, Raidl GR, Soule T, Tyrrell AM, Watson J-P, Zitzler E (eds) GECCO 2005: proceedings of the 2005 conference on genetic and evolutionary computation, vol 2. ACM Press, Washington DC, pp 1851–1858 Drugowitsch J, Barry AM (2005) XCS with eligibility traces. In: Beyer AM, O’Reilly U-M, Arnold DV, Banzhaf W, Blum C, Bonabeau EW, Cantu-Paz E, Dasgupta D, Deb K, Foster JA, de Jong ED, Lipson H, Llora X, Mancoridis S, Pelikan M, Raidl GR, Soule T, Tyrrell AM, Watson J-P, Zitzler E (eds) GECCO 2005: proceedings of the 2005 conference on genetic and evolutionary computation, vol 2. ACM Press, Washington DC, pp 1851–1858
15.
Zurück zum Zitat Holland JH (1976) Adaptation. In: Rosen R, Snell F (eds) Progress in theoretical biology, vol 4. Academic Press, New York, pp 263–293 Holland JH (1976) Adaptation. In: Rosen R, Snell F (eds) Progress in theoretical biology, vol 4. Academic Press, New York, pp 263–293
16.
Zurück zum Zitat Holland JH, Reitman JS (1988) Cognitive systems based on adaptive algorithms, 1978. Reprinted In: David B. Fogel (ed) Evolutionary computation. The fossil record. IEEE Press, New York. ISBN: 0-7803-3481-7 Holland JH, Reitman JS (1988) Cognitive systems based on adaptive algorithms, 1978. Reprinted In: David B. Fogel (ed) Evolutionary computation. The fossil record. IEEE Press, New York. ISBN: 0-7803-3481-7
17.
Zurück zum Zitat Lanzi PL (1997) A study on the generalization capabilities of XCS. In: Baeck T (ed) Proceedings of the seventh international conference on genetic algorithms, 19–23 April east lansing (MI). Morgan Kaufmann, San Francisco, pp 418–425 Lanzi PL (1997) A study on the generalization capabilities of XCS. In: Baeck T (ed) Proceedings of the seventh international conference on genetic algorithms, 19–23 April east lansing (MI). Morgan Kaufmann, San Francisco, pp 418–425
18.
Zurück zum Zitat Lanzi PL (1999) An analysis of generalization in the XCS classifier system. Evol Comput J 7(2):125–149CrossRef Lanzi PL (1999) An analysis of generalization in the XCS classifier system. Evol Comput J 7(2):125–149CrossRef
19.
Zurück zum Zitat Sutton RS, Barto AG (1998) Reinforcement learning—an introduction. MIT Press, Cambridge Sutton RS, Barto AG (1998) Reinforcement learning—an introduction. MIT Press, Cambridge
21.
Zurück zum Zitat Wilson SW (1995) Classifier fitness based on accuracy. Evol Comput 3(2):149–175CrossRef Wilson SW (1995) Classifier fitness based on accuracy. Evol Comput 3(2):149–175CrossRef
22.
Zurück zum Zitat Wilson SW (1998) Generalization in the XCS classifier system. In: Koza JR, Banzhaf W, Chellapilla K, Dorigo KDM, Fogel DB, Garzon MH, Iba DEGH, Riolo R (eds) Genetic programming 1998: proceedings of the third annual conference. Morgan Kaufmann, Menlo Park, pp 665–674. http://prediction-dynamics.com/ Wilson SW (1998) Generalization in the XCS classifier system. In: Koza JR, Banzhaf W, Chellapilla K, Dorigo KDM, Fogel DB, Garzon MH, Iba DEGH, Riolo R (eds) Genetic programming 1998: proceedings of the third annual conference. Morgan Kaufmann, Menlo Park, pp 665–674. http://​prediction-dynamics.​com/​
23.
Zurück zum Zitat Wilson SW (2000) Mining oblique data with XCS. Technical report 2000028, University of Illinois at Urbana-Champaign Wilson SW (2000) Mining oblique data with XCS. Technical report 2000028, University of Illinois at Urbana-Champaign
Metadaten
Titel
Sequential problems that test generalization in learning classifier systems
verfasst von
Martin V. Butz
Pier Luca Lanzi
Publikationsdatum
01.12.2009
Verlag
Springer-Verlag
Erschienen in
Evolutionary Intelligence / Ausgabe 3/2009
Print ISSN: 1864-5909
Elektronische ISSN: 1864-5917
DOI
https://doi.org/10.1007/s12065-009-0019-y

Premium Partner