Skip to main content
Erschienen in: Evolutionary Intelligence 4/2009

01.03.2009 | Research Paper

Automated feature selection in neuroevolution

verfasst von: Maxine Tan, Michael Hartley, Michel Bister, Rudi Deklerck

Erschienen in: Evolutionary Intelligence | Ausgabe 4/2009

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Feature selection is a task of great importance. Many feature selection methods have been proposed, and can be divided generally into two groups based on their dependence on the learning algorithm/classifier. Recently, a feature selection method that selects features at the same time as it evolves neural networks that use those features as inputs called Feature Selective NeuroEvolution of Augmenting Topologies (FS-NEAT) was proposed by Whiteson et al. In this paper, a novel feature selection method called Feature Deselective NeuroEvolution of Augmenting Topologies (FD-NEAT) is presented. FD-NEAT begins with fully connected inputs in its networks, and drops irrelevant or redundant inputs as evolution progresses. Herein, the performances of FD-NEAT, FS-NEAT and traditional NEAT are compared in some mathematical problems, and in a challenging race car simulator domain (RARS). On the whole, the results show that FD-NEAT significantly outperforms FS-NEAT in terms of network performance and feature selection, and evolves networks that offer the best compromise between network size and performance.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Anhänge
Nur mit Berechtigung zugänglich
Literatur
1.
Zurück zum Zitat Zongker D, Jain A (1996) Algorithms for feature selection: an evaluation. In: Proceedings of the 13th international conference on pattern recognition. Vienna, Austria, pp 18–22 Zongker D, Jain A (1996) Algorithms for feature selection: an evaluation. In: Proceedings of the 13th international conference on pattern recognition. Vienna, Austria, pp 18–22
2.
Zurück zum Zitat Kittler J (1978) Feature set search algorithms. In: Chen CH (ed) Pattern recognition and signal processing. Sijthoff and Noordhoff, Alphen aan den Rijn, Netherlands, pp 41–60 Kittler J (1978) Feature set search algorithms. In: Chen CH (ed) Pattern recognition and signal processing. Sijthoff and Noordhoff, Alphen aan den Rijn, Netherlands, pp 41–60
3.
Zurück zum Zitat Mao KZ (2002) Fast orthogonal forward selection algorithm for feature subset selection. IEEE Trans Neural Netw 13:1218–1224CrossRef Mao KZ (2002) Fast orthogonal forward selection algorithm for feature subset selection. IEEE Trans Neural Netw 13:1218–1224CrossRef
4.
Zurück zum Zitat Narendra PM, Fukunaga K (1977) A branch and bound algorithm for feature subset selection. IEEE Trans Comput c-26:917–922CrossRef Narendra PM, Fukunaga K (1977) A branch and bound algorithm for feature subset selection. IEEE Trans Comput c-26:917–922CrossRef
5.
Zurück zum Zitat Ferri FJ, Pudil P, Hatef M, Kittler J (1994) Comparative study of techniques for large-scale feature selection. In: Gelsema ES, Kanal LN (eds) Pattern recognition in practice IV. Elsevier Science B.V., Amsterdam, pp 403–413 Ferri FJ, Pudil P, Hatef M, Kittler J (1994) Comparative study of techniques for large-scale feature selection. In: Gelsema ES, Kanal LN (eds) Pattern recognition in practice IV. Elsevier Science B.V., Amsterdam, pp 403–413
6.
Zurück zum Zitat Pudil P, Novovicova J, Kittler J (1994) Floating search methods in feature selection. Pattern Recognit Lett 15:1119–1125CrossRef Pudil P, Novovicova J, Kittler J (1994) Floating search methods in feature selection. Pattern Recognit Lett 15:1119–1125CrossRef
7.
Zurück zum Zitat Jain A, Zongker D (1997) Feature selection: evaluation, application, and small sample performance. IEEE Trans Pattern Anal Machine Intell 19:153–157CrossRef Jain A, Zongker D (1997) Feature selection: evaluation, application, and small sample performance. IEEE Trans Pattern Anal Machine Intell 19:153–157CrossRef
8.
Zurück zum Zitat Kudo M, Sklansky J (2000) Comparison of algorithms that select features for pattern classifiers. Pattern Recognit 33:25–41CrossRef Kudo M, Sklansky J (2000) Comparison of algorithms that select features for pattern classifiers. Pattern Recognit 33:25–41CrossRef
9.
Zurück zum Zitat Kohavi R, John GH (1997) Wrappers for feature subset selection. Artif Intell 97:273–324MATHCrossRef Kohavi R, John GH (1997) Wrappers for feature subset selection. Artif Intell 97:273–324MATHCrossRef
10.
Zurück zum Zitat Langley P (1994) Selection of relevant features in machine learning. In: Proceedings of the AAAI fall symposium on relevance. AAAI Press, New Orleans Langley P (1994) Selection of relevant features in machine learning. In: Proceedings of the AAAI fall symposium on relevance. AAAI Press, New Orleans
11.
Zurück zum Zitat Bonnlander BV, Weigend AS (1994) Selecting input variables using mutual information and nonparametric density estimation. In: Proceedings of the 1994 international symposium on artificial neural networks (ISANN’94). Tainan, Taiwan, pp 42–50 Bonnlander BV, Weigend AS (1994) Selecting input variables using mutual information and nonparametric density estimation. In: Proceedings of the 1994 international symposium on artificial neural networks (ISANN’94). Tainan, Taiwan, pp 42–50
12.
Zurück zum Zitat Yang J, Honavar V (1998) Feature subset selection using a genetic algorithm. IEEE Intell Syst 13:44–49CrossRef Yang J, Honavar V (1998) Feature subset selection using a genetic algorithm. IEEE Intell Syst 13:44–49CrossRef
13.
Zurück zum Zitat Whiteson S, Stanley KO, Miikkulainen R (2004) Automatic feature selection in neuroevolution. In: Proceedings of the genetic and evolutionary computation conference (GECCO), Seattle, Washington, USA Whiteson S, Stanley KO, Miikkulainen R (2004) Automatic feature selection in neuroevolution. In: Proceedings of the genetic and evolutionary computation conference (GECCO), Seattle, Washington, USA
14.
Zurück zum Zitat Whiteson S, Stone P, Stanley KO, Miikkulainen R, Kohl N (2005) Automatic feature selection in neuroevolution. In: Proceedings of the genetic and evolutionary computation conference (GECCO), Washington, DC, USA Whiteson S, Stone P, Stanley KO, Miikkulainen R, Kohl N (2005) Automatic feature selection in neuroevolution. In: Proceedings of the genetic and evolutionary computation conference (GECCO), Washington, DC, USA
15.
Zurück zum Zitat Stanley KO, Miikkulainen R (2002) Evolving neural networks through augmenting topologies. Evol Comput 10:99–127CrossRef Stanley KO, Miikkulainen R (2002) Evolving neural networks through augmenting topologies. Evol Comput 10:99–127CrossRef
16.
Zurück zum Zitat Stanley KO, Miikkulainen R (2004) Competitive coevolution through evolutionary complexification. J Artif Intell Res 21:63–100 Stanley KO, Miikkulainen R (2004) Competitive coevolution through evolutionary complexification. J Artif Intell Res 21:63–100
17.
Zurück zum Zitat Langley P, Sage S (1994) Oblivious decision trees and abstract cases. In: Working notes of the AAAI-94 workshop on case-based reasoning. AAAI Press, Seattle, pp 113–117 Langley P, Sage S (1994) Oblivious decision trees and abstract cases. In: Working notes of the AAAI-94 workshop on case-based reasoning. AAAI Press, Seattle, pp 113–117
18.
Zurück zum Zitat Kelly JDK, Davis L (1991) Hybridizing the genetic algorithm and the k nearest neighbors classification algorithm. In: Belew RK, Booker LB (eds) Proceedings of the 4th international conference on genetic algorithms. Morgan Kaufmann, San Diego, pp 377–383 Kelly JDK, Davis L (1991) Hybridizing the genetic algorithm and the k nearest neighbors classification algorithm. In: Belew RK, Booker LB (eds) Proceedings of the 4th international conference on genetic algorithms. Morgan Kaufmann, San Diego, pp 377–383
20.
Zurück zum Zitat Gomez F, Miikkulainen R (1998) 2-D pole balancing with recurrent evolutionary networks. In: Proceedings of the international conference on artificial neural networks (ICANN-98), Skovde, Sweden. Elsevier, New York Gomez F, Miikkulainen R (1998) 2-D pole balancing with recurrent evolutionary networks. In: Proceedings of the international conference on artificial neural networks (ICANN-98), Skovde, Sweden. Elsevier, New York
21.
Zurück zum Zitat Gomez F, Miikkulainen R (1997) Incremental evolution of complex general behavior. Adapt Behav 5:317–342CrossRef Gomez F, Miikkulainen R (1997) Incremental evolution of complex general behavior. Adapt Behav 5:317–342CrossRef
22.
Zurück zum Zitat Gruau F, Whitley D, Pyeatt L (1996) A comparison between cellular encoding and direct encoding for genetic neural networks. In: Koza JR, Goldberg DE, Fogel DB, Riolo, RL (eds) Proceedings of the first annual conference on genetic programming, Cambridge, MA, pp 81–89 Gruau F, Whitley D, Pyeatt L (1996) A comparison between cellular encoding and direct encoding for genetic neural networks. In: Koza JR, Goldberg DE, Fogel DB, Riolo, RL (eds) Proceedings of the first annual conference on genetic programming, Cambridge, MA, pp 81–89
23.
Zurück zum Zitat Stanley KO (2004) Efficient evolution of neural networks through complexification. PhD thesis, The University of Texas at Austin Stanley KO (2004) Efficient evolution of neural networks through complexification. PhD thesis, The University of Texas at Austin
24.
Zurück zum Zitat Harvey I (1992) Species adaptation genetic algorithms: a basis for a continuing SAGA. In: Varela FJ, Bourgine P (eds) Proceedings of the 1st European conference on artificial life, toward a practice of autonomous systems. MIT Press/Bradford Books, Cambridge, pp 346–354 Harvey I (1992) Species adaptation genetic algorithms: a basis for a continuing SAGA. In: Varela FJ, Bourgine P (eds) Proceedings of the 1st European conference on artificial life, toward a practice of autonomous systems. MIT Press/Bradford Books, Cambridge, pp 346–354
25.
Zurück zum Zitat Cliff D, Harvey I, Husbands P (1992) Incremental evolution of neural network architectures for adaptive behaviour. Technical report CSRP256, School of Cognitive and Computing Sciences, University of Sussex, UK Cliff D, Harvey I, Husbands P (1992) Incremental evolution of neural network architectures for adaptive behaviour. Technical report CSRP256, School of Cognitive and Computing Sciences, University of Sussex, UK
26.
Zurück zum Zitat Gomez FJ, Miikkulainen R (1999) Solving non-markovian control tasks with neuroevolution. In: Proceedings of the international joint conference on artificial intelligence. Stockholm, Sweden. Morgan Kaufmann, Denver Gomez FJ, Miikkulainen R (1999) Solving non-markovian control tasks with neuroevolution. In: Proceedings of the international joint conference on artificial intelligence. Stockholm, Sweden. Morgan Kaufmann, Denver
27.
Zurück zum Zitat Saravanan N, Fogel DB (1995) Evolving neural control systems. IEEE Expert 10(3):23–27 Saravanan N, Fogel DB (1995) Evolving neural control systems. IEEE Expert 10(3):23–27
28.
Zurück zum Zitat Wieland AP (1991) Evolving neural network controllers for unstable systems. In: Proceedings of the international joint conference on neural networks, Seattle, WA. Piscataway, New Jersey, pp 667–673 Wieland AP (1991) Evolving neural network controllers for unstable systems. In: Proceedings of the international joint conference on neural networks, Seattle, WA. Piscataway, New Jersey, pp 667–673
29.
Zurück zum Zitat Yao X (1999) Evolving artificial neural networks. Proc IEEE 87:1423–1447CrossRef Yao X (1999) Evolving artificial neural networks. Proc IEEE 87:1423–1447CrossRef
30.
Zurück zum Zitat Moriarty DE, Miikkulainen R (1996) Efficient reinforcement learning through symbiotic evolution. Machine Learn 22:11–32 Moriarty DE, Miikkulainen R (1996) Efficient reinforcement learning through symbiotic evolution. Machine Learn 22:11–32
31.
Zurück zum Zitat Radcliffe NJ (1993) Genetic set recombination and its application to neural network topology optimization. Neural Comput Appl 1:67–90MATHCrossRef Radcliffe NJ (1993) Genetic set recombination and its application to neural network topology optimization. Neural Comput Appl 1:67–90MATHCrossRef
32.
Zurück zum Zitat Dasgupta D, McGregor D (1992) Designing application-specific neural networks using the structured genetic algorithm. In: Proceedings of the international conference on combinations of genetic algorithms and neural networks. IEEE Computer Society Press, USA, pp 87–96 Dasgupta D, McGregor D (1992) Designing application-specific neural networks using the structured genetic algorithm. In: Proceedings of the international conference on combinations of genetic algorithms and neural networks. IEEE Computer Society Press, USA, pp 87–96
33.
Zurück zum Zitat Pujol JCF, Poli R (1997) Evolution of the topology and the weights of neural networks using genetic programming with a dual representation. Technical report CSRP-97-7, School of Computer Science, The University of Birmingham, Birmingham B15 2TT, UK Pujol JCF, Poli R (1997) Evolution of the topology and the weights of neural networks using genetic programming with a dual representation. Technical report CSRP-97-7, School of Computer Science, The University of Birmingham, Birmingham B15 2TT, UK
34.
Zurück zum Zitat Gruau F (1993) Genetic synthesis of modular neural networks. In: S. Forrest (ed) Proceedings of the fifth international conference on genetic algorithms. Morgan Kaufmann, San Mateo, CA, pp 318–325 Gruau F (1993) Genetic synthesis of modular neural networks. In: S. Forrest (ed) Proceedings of the fifth international conference on genetic algorithms. Morgan Kaufmann, San Mateo, CA, pp 318–325
35.
Zurück zum Zitat Angeline PJ, Saunders GM, Pollack JB (1993) An evolutionary algorithm that constructs recurrent neural networks. IEEE Trans Neural Netw 5:54–65CrossRef Angeline PJ, Saunders GM, Pollack JB (1993) An evolutionary algorithm that constructs recurrent neural networks. IEEE Trans Neural Netw 5:54–65CrossRef
36.
Zurück zum Zitat Wieland A (1991) Evolving neural network controllers for unstable systems. In: Proceedings of the international joint conference on neural networks. Piscataway, New Jersey, pp 667–673 Wieland A (1991) Evolving neural network controllers for unstable systems. In: Proceedings of the international joint conference on neural networks. Piscataway, New Jersey, pp 667–673
37.
Zurück zum Zitat Stanley KO, Bryant BD, Miikkulainen R (2005) Real-time neuroevolution in the NERO video game. IEEE Trans Evol Comput 9:653–668CrossRef Stanley KO, Bryant BD, Miikkulainen R (2005) Real-time neuroevolution in the NERO video game. IEEE Trans Evol Comput 9:653–668CrossRef
38.
Zurück zum Zitat Stanley KO, Miikkulainen R (2004) Evolving a roving eye for go. In: Proceedings of the genetic and evolutionary computation conference (GECCO). Springer, New York, pp 1226–1238 Stanley KO, Miikkulainen R (2004) Evolving a roving eye for go. In: Proceedings of the genetic and evolutionary computation conference (GECCO). Springer, New York, pp 1226–1238
39.
Zurück zum Zitat Schlessinger E, Bentley PJ, Lotto RB (2005) Analysing the evolvability of neural network agents through structural mutations. In: Capcarrere M (ed) Proceedings of the European conference on artificial life (ECAL 2005). Springer, Berlin, pp 312–321 Schlessinger E, Bentley PJ, Lotto RB (2005) Analysing the evolvability of neural network agents through structural mutations. In: Capcarrere M (ed) Proceedings of the European conference on artificial life (ECAL 2005). Springer, Berlin, pp 312–321
40.
Zurück zum Zitat Yao X, Liu Y (1996) Towards designing artificial neural networks by evolution. Appl Math Comput 91:83–90CrossRef Yao X, Liu Y (1996) Towards designing artificial neural networks by evolution. Appl Math Comput 91:83–90CrossRef
41.
Zurück zum Zitat Kohl N, Stanley K, Miikkulainen R, Samples M, Sherony R (2006) Evolving a real-world vehicle warning system. In: Proceedings of the genetic and evolutionary computation conference (GECCO), Seattle, Washington, USA, pp 1681–1688 Kohl N, Stanley K, Miikkulainen R, Samples M, Sherony R (2006) Evolving a real-world vehicle warning system. In: Proceedings of the genetic and evolutionary computation conference (GECCO), Seattle, Washington, USA, pp 1681–1688
42.
Zurück zum Zitat Gomez F, Miikkulainen R (1998) 2-D pole balancing with recurrent evolutionary networks. In: Proceedings of the international conference on artificial neural networks (ICANN). Skovde, Sweden. Elsevier, New York Gomez F, Miikkulainen R (1998) 2-D pole balancing with recurrent evolutionary networks. In: Proceedings of the international conference on artificial neural networks (ICANN). Skovde, Sweden. Elsevier, New York
43.
Zurück zum Zitat Goldberg DE, Richardson J (1987) Genetic algorithms with sharing for multimodal function optimization. In: Proceedings of the second international conference on genetic algorithms. Lawrence Erlbaum Associates, Hillsdale, pp 41–49 Goldberg DE, Richardson J (1987) Genetic algorithms with sharing for multimodal function optimization. In: Proceedings of the second international conference on genetic algorithms. Lawrence Erlbaum Associates, Hillsdale, pp 41–49
44.
Zurück zum Zitat Gruau F, Whitley D, Pyeatt L (1996) A comparison between cellular encoding and direct encoding for genetic neural networks. Technical Report NC-TR-96-048, NeuroCOLT Gruau F, Whitley D, Pyeatt L (1996) A comparison between cellular encoding and direct encoding for genetic neural networks. Technical Report NC-TR-96-048, NeuroCOLT
45.
Zurück zum Zitat Gomez F, Miikkulainen R (1999) Solving non-Markovian control tasks with neuroevolution. In: Proceedings of the 16th international joint conference on artificial intelligence. Morgan Kaufmann, Denver Gomez F, Miikkulainen R (1999) Solving non-Markovian control tasks with neuroevolution. In: Proceedings of the 16th international joint conference on artificial intelligence. Morgan Kaufmann, Denver
46.
Zurück zum Zitat Coons KE, Robatmili B, Taylor ME, Maher BA, Burger D, McKinley KS (2008) Feature selection and policy optimization for distributed instruction placement using reinforcement learning. In: Proceedings of the 7th international joint conference on parallel architectures and compilation techniques (PACT), Toronto, Ontario, Canada Coons KE, Robatmili B, Taylor ME, Maher BA, Burger D, McKinley KS (2008) Feature selection and policy optimization for distributed instruction placement using reinforcement learning. In: Proceedings of the 7th international joint conference on parallel architectures and compilation techniques (PACT), Toronto, Ontario, Canada
47.
Zurück zum Zitat Skalak DB (1994) Prototype and feature selection by sampling and random mutation hill-climbing algorithms. In: Proceedings of the 11th international conference on machine learning. Morgan Kaufmann, New Brunswick, pp 293–301 Skalak DB (1994) Prototype and feature selection by sampling and random mutation hill-climbing algorithms. In: Proceedings of the 11th international conference on machine learning. Morgan Kaufmann, New Brunswick, pp 293–301
49.
Zurück zum Zitat Ethembabaoglu A, Whiteson S (2008) Automatic feature selection using FS-NEAT. Technical report IAS-UVA-08-02, Intelligent Autonomous Systems Group, University of Amsterdam Ethembabaoglu A, Whiteson S (2008) Automatic feature selection using FS-NEAT. Technical report IAS-UVA-08-02, Intelligent Autonomous Systems Group, University of Amsterdam
Metadaten
Titel
Automated feature selection in neuroevolution
verfasst von
Maxine Tan
Michael Hartley
Michel Bister
Rudi Deklerck
Publikationsdatum
01.03.2009
Verlag
Springer-Verlag
Erschienen in
Evolutionary Intelligence / Ausgabe 4/2009
Print ISSN: 1864-5909
Elektronische ISSN: 1864-5917
DOI
https://doi.org/10.1007/s12065-009-0018-z

Premium Partner