Top

Neural Computing and Applications

Published in:

01-01-2013 | Cont. Dev. of Neural Compt. & Appln.

ARTIE and MUSCLE models: building ensemble classifiers from fuzzy ART and SOM networks

Authors: César L. C. Mattos, Guilherme A. Barreto

Published in: Neural Computing and Applications | Issue 1/2013

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Ensemble Learning has proven to be an efficient method to improve the performance of single classifiers. In this context, the present article introduces ARTIE (ART networks in Ensembles) and MUSCLE (Multiple SOM Classifiers in Ensembles), two novel ensemble models that use Fuzzy ART and SOM networks as base classifiers, respectively. In addition, a hybrid metaheuristic solution based on Particle Swarm Optimization and Simulated Annealing is used for parameter tuning of the base classifiers. A comprehensive performance comparison using 10 benchmarking data sets indicates that the ARTIE and MUSCLE architectures consistently outperform ensembles built from standard supervised neural networks, such as the Fuzzy ARTMAP, Learning Vector Quantization, and the Extreme Learning Machine.

previous article New conditions for global exponential stability of continuous-time neural networks with delays

next article Learning gene regulatory networks using the bees algorithm

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Bagging is carried out by sampling (with replacement) training examples, forming new training sets, usually with the same size of the original one. For a training set of N samples and N being large enough, this procedure causes each sample to have a probability of \(\left(\frac{N-1}{N}\right)^N \approx 0.368\) of not being chosen.

Barreto GA, Aguayo L (2009) Time series clustering for anomaly detection using competitive neural networks. In: Principe JC, Miikkulainen R (eds) Proceedings of the 7th international workshop on advances in self-organizing maps (WSOM’09), Springer, vol LNCS-5629, pp 28–36

Baruque B, Corchado E (2010) A weighted voting summarization of SOM ensembles. Data Min Knowl Discov 21(3):398–426MathSciNetCrossRef

Bermejo S, Cabestany J (2004) Local averaging of ensembles of LVQ-based nearest neighbor classifiers. Appl Intell 20(1):47–58MATHCrossRef

Biebelmann E, Köppen M, Nickolay B (1996) Pratical aplications of neural networks in texture analysis. Neurocomputing 13(2–4):261–279CrossRef

Biehl M, Ghosh A, Hammer B (2007) Dynamics and generalization ability of LVQ algorithms. J Mach Learn Res 8(Feb):323–360MathSciNetMATH

Bratton D, Kennedy J (2007) Defining a standard for particle swarm optimization. In: Proceedings of the IEEE Swarm Intelligence Symposium, Honolulu, Hawaii, pp 120–127

Breiman L (1996) Bagging predictors. Mach Learn 24(2):123–140MathSciNetMATH

Carpenter G (2003) Default ARTMAP. CAS/CNS Technical Report Series (008)

Carpenter G, Grossberg S (1987) Stable self-organization of pattern recognition codes for analog input patterns. Appl Opt 26:4919–4930CrossRef

10.

Carpenter G, Grossberg S, Reynolds JH (1991) ARTMAP: supervised real-time learning and classification of nonstationary data by a self-organizing neural network. Neural Netws 4(5):565–588CrossRef

11.

Carpenter GA, Grossberg S (1987) A massively parallel architecture for a self-organizing neural pattern recognition machine. Comput Vis Graph Image Process 37(1):54–115MATHCrossRef

12.

Carpenter GA, Grossberg S, Rosen DB (1991) Fuzzy ART: fast stable learning, categorization of analog patterns by an adaptive resonance system. Neural Netw 4(6):759–771CrossRef

13.

Carpenter GA, Grossberg S, Markuzon N, Reynolds JH, Rosen DB (2002) Fuzzy ARTMAP: a neural network architecture for incremental supervised learning of analog multidimensional maps. IEEE Trans Neural Netw 3(5):698–713CrossRef

14.

Chang Y, Lee DJ, Hong Y, Archibald J (2008) Unsupervised video shot detection using clustering ensemble with a color global scale-invariant feature transform descriptor. EURASIP J Image Video Process 2008(Article ID 860743):10 pages. doi:10.1155/2008/860743

15.

Cho SB (1997) Self-organizing map with dynamical node splitting: application to handwritten digit recognition. Neural Comput 9(6):1345–1355CrossRef

16.

Christodoulou CI, Michaelides SC, Pattichis CS (2003) Multifeature texture analysis for the classification of clouds in satellite imagery. IEEE Trans Geosci Remote Sens 41(11):2662–2668CrossRef

17.

Corchado E, Baruque B, Yin H (2007) Boosting unsupervised competitive learning ensembles. In: de Sá JM, Alexandre LA, Duch W, Mandic DP (eds) Proceedings of the 17th international conference on artificial neural networks (ICANN’07), part I, Springer, vol LNCS 4668, pp 339–348

18.

del-Hoyo R, Buldain D, Marco A (2003) Supervised classification with associative SOM. In: Proceedings of the 7th international work-conference on artificial and neural networks, (IWANN)’03, pp 334–341

19.

Dietterich TG (1998) Approximate statistical tests for comparing supervised classification learning algorithms. Neural Comput 10(7):1895–1923CrossRef

20.

Dietterich TG (2000) An experimental comparison of three methods for constructing ensembles of decision trees: bagging, boosting, and randomization. Mach Learn 40(2):139–157CrossRef

21.

Dietterich TG (2003) Ensemble learning. In: Arbib MA (ed) The handbook of brain theory, neural networks, 2nd edn. MIT Press, Cambridge, pp 405–408

22.

Everitt B (1977) The analysis of contingency tables. Chapman & Hall, CRC, London

23.

Frank A, Asuncion A (2010) UCI machine learning repository. http://www.archive.ics.uci.edu/ml

24.

Georgakis A, Li H, Gordan M (2005) An ensemble of SOM networks for document organization and retrieval. In: International conference on adaptive knowledge representation and reasoning (AKRR’05), pp 141–147

25.

Gorgonio FA, Costa JAF (2008) Parallel self-organizing maps with application in clustering distributed data. In: Proceedings of the 2008 IEEE international joint conference on neural networks (IJCNN’2008), pp 3276–3283

26.

Hansen LK, Salamon P (2002) Neural network ensembles. IEEE Trans Pattern Anal Mach Intell 12(10):993–1001CrossRef

27.

Haykin S (2008) Neural networks and learning machines, 3rd edn. Prentice Hall, NJ

28.

He Q, Wang L (2007) A hybrid particle swarm optimization with a feasibility-based rule for constrained optimization. Appl Math Comput 186(2):1407–1422MathSciNetMATHCrossRef

29.

Huang CL, Dun J (2008) A distributed PSO-SVM hybrid system with feature selection and parameter optimization. Appl Soft Comput 8(4):1381–1391CrossRef

30.

Huang GB, Zhu QY, Ziew CK (2006) Extreme learning machine: theory and applications. Neurocomputing 70(1–3):489–501CrossRef

31.

Huynh H, Yonggwan W, Kim J (2008) An improvement of extreme learning machine for compact single-hidden-layer feedforward neural networks. Int J Neural Syst 18(5):433–441CrossRef

32.

Jiang Y, Zhou Z (2004) SOM ensemble-based image segmentation. Neural Process Lett 20(3):171–178CrossRef

33.

Kangas JA, Kohonen TK, Laaksonen JT (1990) Variants of self-organizing maps. IEEE Trans Neural Netw 1(1):93–99CrossRef

34.

Kennedy J, Eberhart RC (1995) Particle swarm optimization. In: Proceedings of the IEEE international conference on neural networks, Piscataway, NJ, USA, vol 4, pp. 1942–1948

35.

Keskin GA, Özkan C (2009) An alternative evaluation of FMEA: fuzzy art algorithm. Qual Reliabil Eng Int 25(6):647–661CrossRef

36.

Kohonen T (1982) Self-organized formation of topologically correct feature maps. Biol Cybernet 43(1):59–69MathSciNetMATHCrossRef

37.

Kohonen T (1988) An introduction to neural computing. Neural Netw 1(1):3–16CrossRef

38.

Kohonen T (1988) The ’neural’ phonetic typewriter. Computer 21(3):11–22CrossRef

39.

Kohonen T (1990) The self-organizing map. Proc of the IEEE 78(9):1464–1480CrossRef

40.

Kohonen T (2003) Learning vector quantization. In: Arbib MA (ed) The handbook of brain theory, neural networks, 2nd edn. MIT Press, Cambridge, pp 631–635

41.

Krogh A, Vedelsby J (1995) Neural network ensembles, cross validation, active learning. Advances in neural information processing systems, pp. 231–238

42.

Laha A, Pal NR (2001) Some novel classifiers designed using prototypes extracted by a new scheme based on self-organizing feature map. IEEE Trans Syst Man Cybern B 31(6):881–890CrossRef

43.

Liu N, Wang H (2010) Ensemble based extreme learning machine. IEEE Signal Process Lett 17(8):754–757CrossRef

44.

Loo CK, Law A, Lim WS, Rao MVC (2006) Probabilistic ensemble simplified fuzzy ARTMAP for sonar target differentiation. Neural Comput Appl 15(1):79–90CrossRef

45.

Madeo R, Peres SM, Bíscaro HH, Dias DB, Boscarioli C (2010) A committee machine implementing the pattern recognition module for fingerspelling applications. In: Proceedings of the 2010 ACM symposium on applied computing (SAC’2010), pp 954–958

46.

Martinetz TM, Schulten KJ (1991) A “neural-gas” network learns topologies. In: Kohonen T, Makisara K, Simula O, Kangas J (eds) Artificial neural networks. North-Holland, Amsterdam, pp 397–402

47.

Monteiro IQ, Queiroz SA, Carneiro AT, Souza LGM, Barreto GA (2006) Face recognition independent of facial expression through SOM-based classifiers. In: Proceedings of the IEEE/SBrT international telecommunications symposium (ITS’06), Fortaleza, Ceará, Brazil, pp 263–268

48.

Pedersen MEH, Chipperfield AJ (2010) Simplifying particle swarm optimization. Appl Soft Comput 10(2):618–628CrossRef

49.

Pedreira CE (2006) Learning vector quantization with training data selection. IEEE Trans Pattern Anal Mach Intell 28(1):157–162CrossRef

50.

Petrikieva L, Fyfe C (2002) Bagging and bumping self-organising maps. Comput Inform Syst 9(2):69

51.

Platt JC (1998) Fast training of support vector machines using sequential minimal optimization. In: Advances in Kernel methods: support vector learning. MIT Press, Cambridge, pp 185–208

52.

Raafat HM, Tolba AS, Aly AM (2011) A novel training weighted ensemble (TWE) with application to face recognition. Appl Soft Comp 3608–3617

53.

Rocha M, Cortez P, Neves J (2007) Evolution of neural networks for classification and regression. Neurocomputing 70(16–18):1054–1060

54.

Rocha Neto AR, Barreto GA (2009) On the application of ensembles of classifiers to the diagnosis of pathologies of the vertebral column: a comparative analysis. IEEE Latin Am Trans 7(4):487–496CrossRef

55.

Rokach L (2010) Ensemble-based classifiers. Artif Intell Rev 33(1-2):1–39CrossRef

56.

Santos AM, Canuto AMP (2008) Using ARTMAP-based ensemble systems designed by three variants of boosting. In: Proceedings of the international conference on artificial neural networks (ICANN’08), pp 562–571

57.

Sartain P, Hopkins ABT, McDonald-Mair KD, Howells WGJ (2008) A framework for self-diagnosis, condition monitoring for embedded systems using a SOM-based classifier. In: NASA/ESA conference on adaptive hardware, systems 2008, IEEE, pp 417–423

58.

Scherbart A, Nattkemper TW (2011) Looking inside self-organizing map ensembles with resampling and negative correlation learning. Neural Netw 24(1):130–141MATHCrossRef

59.

Souza~Júnior AH, Barreto GA, Varela AT (2011) A speech recognition system for embedded applications using the SOM and TS-SOM networks. In: Mwasiagi JI (ed) Self-organizing maps—applications and novel algorithm design. InTech Open, Croatia, pp 97–108

60.

Suganthan PN (1999) Hierarchical overlapped SOM’s for pattern classification. IEEE Trans Neural Netw 10(1):193–196CrossRef

61.

Turky AM, Ahmad MS (2010) The use of SOM for fingerprint classification. In: IEEE international conference on information retrieval & knowledge management (CAMP’2010), pp 287–290

62.

Umer MF, Khiyal MSH (2007) Classification of textual documents using learning vector quantization. Inform Technol J 6:154–159CrossRef

63.

Wyns B, Sette S, Boullart L, Baeten D, Hoffman IEA, De Keyser F (2004) Prediction of diagnosis in patients with early arthritis using a combined Kohonen mapping and instance-based evaluation criterion. Artif Intell Med 31(1):45–55CrossRef

64.

Wyns B, Sette S, Boullart L, Baeten D, Hoffman IEA, Keyser FD (2004) Prediction of diagnosis in patients with early arthritis using a combined Kohonen mapping, instance-based evaluation criterion. Artif Intell Med 31(1):45–55CrossRef

65.

Xiao YD, Clauset A, Harris R, Bayram E, Santago P, Schmitt JD (2005) Supervised self-organizing maps in drug discovery. 1. Robust behavior with overdetermined data sets. J Chem Inform Model 45(6):1749–1758CrossRef

Title: ARTIE and MUSCLE models: building ensemble classifiers from fuzzy ART and SOM networks
Authors: César L. C. Mattos
Guilherme A. Barreto
Publication date: 01-01-2013
Publisher: Springer-Verlag
Published in: Neural Computing and Applications / Issue 1/2013
Print ISSN: 0941-0643
Electronic ISSN: 1433-3058
DOI: https://doi.org/10.1007/s00521-011-0747-7

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Springer Professional "Wirtschaft+Technik"

Other articles of this Issue 1/2013

Synchronization criteria and pinning control for complex networks with multiple delays

Prediction of the effect of varying cure conditions and w/c ratio on the compressive strength of concrete using artificial neural networks

A novel weight pruning method for MLP classifiers based on the MAXCORE principle

Reducing blocking artifacts in JPEG-compressed images using an adaptive neural network-based algorithm

Research of assembling optimized classification algorithm by neural network based on Ordinary Least Squares (OLS)

Stability of Hopfield neural networks with time delays and variable-time impulses

Premium Partner