Top

Soft Computing

Published in:

01-04-2010 | Original Paper

Designing multilayer perceptrons using a Guided Saw-tooth Evolutionary Programming Algorithm

Authors: Pedro Antonio Gutiérrez, César Hervás, Manuel Lozano

Published in: Soft Computing | Issue 6/2010

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

In this paper, a diversity generating mechanism is proposed for an Evolutionary Programming (EP) algorithm that determines the basic structure of Multilayer Perceptron classifiers and simultaneously estimates the coefficients of the models. We apply a modified version of a saw-tooth diversity enhancement mechanism recently presented for Genetic Algorithms, which uses a variable population size and periodic partial reinitializations of the population in the form of a saw-tooth function. Our improvement on this standard scheme consists of guiding saw-tooth reinitializations by considering the variance of the best individuals in the population. The population restarts are performed when the difference of variance between two consecutive generations is lower than a percentage of the previous variance. From the analysis of the results over ten benchmark datasets, it can be concluded that the computational cost of the EP algorithm with a constant population size is reduced by using the original saw-tooth scheme. Moreover, the guided saw-tooth mechanism involves a significantly lower computer time demand than the original scheme. Finally, both saw-tooth schemes do not involve an accuracy decrease and, in general, they obtain a better or similar precision.

previous article Multiobjective evolutionary clustering of Web user sessions: a case study in Web page recommendation

next article Monitoring defects of ceramic tiles using fuzzy subtractive clustering-based system identification method

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Alcala-Fdez J, Sánchez L, García S, del Jesús MJ, Ventura S, Garrell JM, Otero J, Romero C, Bacardit J, Rivas VM, Fernández JC, Herrera F (2008) KEEL: a software tool to assess evolutionary algorithms for data mining problems. Soft Comput 13(3):307–318CrossRef

Angeline PJ, Sauders GM, Pollack JB (1994) An evolutionary algorithm that constructs recurrent neural networks. IEEE Trans Neural Netw 5:54–65CrossRef

Asuncion A, Newman D (2007) UCI machine learning repository. http://www.ics.uci.edu/mlearn/MLRepository.html

Back T, Fogel DB, Michalewicz Z (1997) Handbook of evolutionary computation. IOP Publishing Ltd, BristolCrossRef

Bishop CM (2006) Pattern recognition and machine learning. Springer, Berlin

Bornholdt S, Graudenz D (1992) General asymmetric neural networks and structure design by genetic algorithms. Neural Netw 5(2):327–334. doi:10.1016/S0893-6080(05)80030-9

Bramlette MF (1991) Initialization, mutation and selection methods in genetic algorithms for function optimization. In: Belew RK, Booker LB (eds) Proceedings of the 4th international conference on genetic algorithms (ICGA’91), pp 100–107

Chaiyaratana N, Piroonratana T, Sangkawelert N (2007) Effects of diversity control in single-objective and multi-objective genetic algorithms. J Heuristics 13(1):1–34CrossRef

Cobb HG, Grefenstette JJ (1993) Genetic algorithms for tracking changing environments. In: Proceedings of the 5th international conference on genetic algorithms, pp 523–530

De Jong ED, Watson RA, Pollack JB (2001) Reducing bloat and promoting diversity using multi-objective methods. In: Proceedings of the 2001 genetic and evolutionary computation conference, Morgan Kaufmann, San Francisco, CA, pp 11–18

De Jong KA (1975) An analysis of the behavior of a class of genetic adaptive systems. PhD thesis, University of Michigan, MI, USA

Eiben AE, Smith JE (2003) Introduction to evolutionary computing (Natural Computing Series). Springer, Berlin

Eshelman LJ (1991) The CHC adaptive search algorithm: How to have safe search when engaging in nontraditional genetic recombination. In: Proceedings of the 1st workshop on foundations on genetic algorithms, pp 257–266

Eshelman LJ, Schaffer JD (1991) Preventing premature convergence in genetic algorithms by preventing incest. In: Belew R, Booker L (eds) Proceedings of the fourth international conference on genetic algorithms, Morgan Kaufman, San Mateo, CA, pp 115–122

Fogel D (1995) Evolutionary computation: toward a new philosophy of machine intelligence. IEEE Press, New York

Fogel D, Owens A, Wals MJ (1966) Artificial intelligence through simulated evolution. Wiley, New YorkMATH

Fogel DB, Fogel LJ, Porto VW (1990) Evolving neural networks. Biol Cybern 63(6):487–493. 10.1007/BF00199581

Goldberg DE (1989) Genetic algorithms in search, optimization, and machine learning. Addison-Wesley, Reading

Goldberg DE, Deb K (1990) A comparative analysis of selection schemes used in genetic algorithms. In: Proceedings of the first workshop on foundations of genetic algorithms, Bloomington Campus, Indiana, USA, pp 69–93

Goldberg DE, Richardson J (1987) Genetic algorithms with sharing for multi-modal function optimization. In: Proceedings of the 2nd international conference on genetic algorithms, Lawrence Erlbaum Associates, pp 41–49

Goldberg DE, Deb K, Clark JH (1992) Genetic algorithms, noise, and the sizing of populations. Complex Syst 6(4):333–362MATH

Greewood GW, Fogel GB, Ciobanu M (1999) Emphasizing extinction in evolutionary programming. In: Angeline P, Michalewicz Z, Schoenauer M, Yao X, Zalzala A (eds) Proceedings of the 1999 congress on evolutionary computation (CEC99), pp 666–671

Guillén A, Rojas I, González J, Pomares H, Herrera L, Valenzuela O, Rojas F (2007) Output value-based initialization for radial basis function neural networks. Neural Process Lett 25(3):209–225CrossRef

Gutiérrez PA, Hervás-Martínez C, Lozano M (2007) Saw-tooth algorithm guided by the variance of best individual distributions for designing evolutionary neural networks. In: Proceedings of the 8th international conference on intelligent data engineering and automated learning (IDEAL’07), Springer, Birmingham, Lecture Notes in Computer Science, vol 4881, pp 1131–1140

Herrera F, Lozano M (1996) Adaptation of genetic algorithm parameters based on fuzzy logic controllers. In: Herrera F, Verdegay JL (eds) Genetic algorithms and soft computing, Physica-Verlag, Heidelberg, pp 95–125

Herrera F, Lozano M, Verdegay JL (1996) Dynamic and heuristic fuzzy connectives-based crossover operators for controlling the diversity and convergence of real-coded genetic algorithms. Int J Intell Syst 11(12):1013–1040MATHCrossRef

Hervás C, Martínez-Estudillo FJ, Gutiérrez PA (2006) Classification by means of evolutionary product-unit neural networks. In: Proceedings of the 2006 international joint conference on neural networks, Vancouver, Canada, pp 2834–2842

Hutter M, Legg S (2006) Fitness uniform optimization. IEEE Trans Evol Comput 10(5):568–589CrossRef

Koumousis VK, Dimou CK (2002) The effect of oscillating population size on the performance of genetic algorithms. In: Proceedings of the 4th GRACM congress on computational mechanics

Koumousis VK, Katsaras CP (2006) A saw-tooth genetic algorithm combining the effects of variable population size and reinitialization to enhance performance. IEEE Trans Evol Comput 10(1):19–28CrossRef

Krishnakumar K (1989) Micro-genetic algorithms for stationary and non-stationary function optimization. In: Proceedings of the SPIE: intelligent control and adaptive systems, vol 1196, pp 289–296

Landwehr N, Hall M, Frank E (2005) Logistic model trees. Mach Learn 59(1–2):161–205MATHCrossRef

Leung FHF, Lam HK, Ling SH, Tam PKS (2003) Tuning of the structure and parameters of a neural network using an improved genetic algorithm. IEEE Trans Neural Netw 14(1):79–88. doi:10.1109/TNN.2002.804317

Levene H (1960) Essays in Honor of Harold Hotelling. In: Contributions to probability and statistics. Stanford University Press, London

Lozano M, Herrera F, Cano JR (2008) Replacement strategies to preserve useful diversity in steady-state genetic algorithms. Inf Sci 178(23):4421–4433CrossRef

Mahfoud SW (1992) Crowding and preselection revisited. In: Männer R, Manderick B (eds) Parallel problem solving from nature 2, North-Holland, Amsterdam, pp 27–36

Maniezzo V (1993) Searching among search spaces: hastening the genetic evolution of feedforward networks. In: Proceedings of the conference on artificial neural nets and genetic algorithms

Martínez-Estudillo AC, Hervás-Martínez C, Martínez-Estudillo FJ, García-Pedrajas N (2006a) Hybridization of evolutionary algorithms and local search by means of a clustering method. IEEE Trans Syst Man Cybern B 36(3):534–545CrossRef

Martínez-Estudillo AC, Martínez-Estudillo FJ, Hervás-Martínez C, García N (2006b) Evolutionary product unit based neural networks for regression. Neural Netw 19(4):477–486MATHCrossRef

Martínez-Estudillo FJ, Hervás-Martínez C, Gutiérrez PA, Martínez-Estudillo AC (2008) Evolutionary product-unit neural networks classifiers. Neurocomputing 72(1–2):548–561CrossRef

McDonnell JR, Waagen DE (1993) Evolving recurrent perceptrons. In: Ruck DW (ed) Proceedings of the SPIE conference, Science of Artificial Neural Networks II, vol 1966, pp 350–361

Miller RG (1986) Beyond ANOVA, basics of applied statistics. Wiley, London

Oppacher F, Wineberg M (1999) The shifting balance genetic algorithm: improving the GA in a dynamic environment. In: Proceedings of the genetic and evolutionary computation conference, Morgan Kaufmann, San Francisco, vol 1, pp 504–510

Palmes PP, Hayasaka T, Usui S (2005) Mutation-based genetic neural network. IEEE Trans Neural Netw 16(3):587–600CrossRef

Prechelt L (1994) PROBEN1: a set of neural network benchmark problems and benchmarking rules. Tech. Rep. 21/94, Fakultät für Informatik (Universität Karlsruhe)

Prügel-Bennett A, Shapiro JL (1994) An analysis of genetic algorithms using statistical mechanics. Phys Rev Lett 72(9):1305–1309CrossRef

Prügel-Bennett A, Shapiro JL (1997) The dynamics of a genetic algorithm for simple Ising systems. Physica D 104:75–114CrossRefMathSciNet

Rattray M (1996) Modeling the dynamics of genetic algorithms using statistical mechanics. PhD thesis, Manchester University, Manchester, UK

Rechenberg I (1973) Evolutionsstrategie: optimierung technischer systeme nach prinzipien der biologischen evolution. Frommann-Holzboog, Stuttgart

Rogers A, Pruegel-Bennett A (1999) Genetic drift in genetic algorithm selection schemes. IEEE Trans Evol Comput 3(4):298CrossRef

Shimodaira H (2001) A diversity-control-oriented genetic algorithm DCGA: performance in function optimization. In: Proceedings of the 2001 congress on evolutionary computation, 2001, vol 1, pp 44–51. doi:10.1109/CEC.2001.934369

Smith RE (1993) Adaptively resizing populations: an algorithm and analysis. In: Proceedings of the fifth international conference on genetic algorithms, pp 653–653

Snedecor GW, Cochran WG (1989) Statistical methods, 6th edn. Iowa State University Press

Tsai JT, Chou JH, Liu TK (2006) Tuning the structure and parameters of a neural network by using hybrid taguchi-genetic algorithm. IEEE Trans Neural Netw 17(1):69–80CrossRef

Tsutsui S, Fujimoto Y, Ghosh A (1997) Forking genetic algorithms: GAs with search space division schemes. Evol Comput 5(1):61–80CrossRef

Ursem RK (2002) Diversity-guided evolutionary algorithms. In: Proceedings of parallel problem solving from nature VII (PPSN-2002), Springer, Berlin, pp 462–471

Ventura S, Romero C, Zafra A, Delgado JA, Hervás C (2008) JCLEC: a Java framework for evolutionary computation. Soft Comput 12(4):381–392CrossRef

Wang L, Zheng DZ, Tang F (2002) An improved evolutionary programming for optimization. In: Proceedings of the 4th World Congress on Intelligent Control and Automation, vol 3, pp 1769–1773

Whitley D (1989) The GENITOR algorithm and selection pressure: why rank-based allocation of reproductive trials is best. In: Schaffer JD (ed) Proceedings of the third international conference on genetic algorithms, Morgan Kaufman, San Mateo, CA

Wolpert DH, Macready WG (1997) No free lunch theorems for optimization. IEEE Trans Evol Comput 1(1):67–82CrossRef

Yao X (1997) Global optimization by evolutionary algorithms. In: Proceedings of the 2nd Aizu international symposium on parallel algorithms/architecutre synthesis, (pAs’97), Aizu-Wakamatsu, Japan, pp 282–291

Yao X (1999) Evolving artificial neural networks. Proc IEEE 87(9):1423–1447CrossRef

Yao X, Liu Y (1997) A new evolutionary system for evolving artificial neural networks. IEEE Trans Neural Netw 8(3):694–713CrossRefMathSciNet

Title: Designing multilayer perceptrons using a Guided Saw-tooth Evolutionary Programming Algorithm
Authors: Pedro Antonio Gutiérrez
César Hervás
Manuel Lozano
Publication date: 01-04-2010
Publisher: Springer-Verlag
Published in: Soft Computing / Issue 6/2010
Print ISSN: 1432-7643
Electronic ISSN: 1433-7479
DOI: https://doi.org/10.1007/s00500-009-0429-x

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Other articles of this Issue 6/2010

IF-topologies and IF-automata

Monitoring defects of ceramic tiles using fuzzy subtractive clustering-based system identification method

Adaptive output recurrent cerebellar model articulation controller for nonlinear system control

On the priority vector associated with a reciprocal relation and a pairwise comparison matrix

Human face recognition using fuzzy multilayer perceptron

Numerical solution of fuzzy relational equations based on smooth fuzzy norms

Premium Partner