Skip to main content
Erschienen in: Neural Computing and Applications 7-8/2013

01.06.2013 | Original Article

Fuzzy rules extraction from support vector machines for multi-class classification

verfasst von: Adriana da Costa F. Chaves, Marley Maria B. R. Vellasco, Ricardo Tanscheit

Erschienen in: Neural Computing and Applications | Ausgabe 7-8/2013

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

This paper proposes a new method for fuzzy rule extraction from trained support vector machines (SVMs) for multi-class problems, named FREx_SVM. SVMs have been used in a variety of applications. However, they are considered “black box models,” where no interpretation about the input–output mapping is provided. Some methods to reduce this limitation have already been proposed, but they are restricted to binary classification problems and to the extraction of symbolic rules with intervals or functions in their antecedents. In order to improve the interpretability of the generated rules, this paper presents a new model for extracting fuzzy rules from a trained SVM. The proposed model is suited for classification in multi-class problems and includes a wrapper feature selection algorithm. It is evaluated in four benchmark databases, and results obtained demonstrate its capacity to generate a reduced set of interpretable fuzzy rules that explains both the classification database and the influence of each input variable on the determination of the final class.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Abe S, Inoue T (2002) Fuzzy support vector machines for multiclass problems, In: European symposium on artificial neural networks proceedings (ESANN), pp 113–118 Abe S, Inoue T (2002) Fuzzy support vector machines for multiclass problems, In: European symposium on artificial neural networks proceedings (ESANN), pp 113–118
2.
Zurück zum Zitat Barakat N, Bradley A (2010) Rule extraction from support vector machines: a review. Neurocomputing 74:178–190CrossRef Barakat N, Bradley A (2010) Rule extraction from support vector machines: a review. Neurocomputing 74:178–190CrossRef
3.
Zurück zum Zitat Barakat N, Diederich J (2004) Learning-based rule-extraction from support vector machines: performance on benchmark data sets. In: Proceedings of the conference on neuro-computing and evolving intelligence, Knowledge Engineering and Discovery Research Institute (KEDRI), Auckland, New Zealand Barakat N, Diederich J (2004) Learning-based rule-extraction from support vector machines: performance on benchmark data sets. In: Proceedings of the conference on neuro-computing and evolving intelligence, Knowledge Engineering and Discovery Research Institute (KEDRI), Auckland, New Zealand
4.
Zurück zum Zitat Barakat N, Diederich J (2005) Eclectic rule-extraction from support vector machines. Int J Comput Intell 2:59–62 Barakat N, Diederich J (2005) Eclectic rule-extraction from support vector machines. Int J Comput Intell 2:59–62
5.
Zurück zum Zitat Castro J, Flores-Hidalgo L, Mantas C, Puche J (2007) Extraction of fuzzy rules from support vector machines. Fuzzy Sets Syst 158:2057–2077MathSciNetMATHCrossRef Castro J, Flores-Hidalgo L, Mantas C, Puche J (2007) Extraction of fuzzy rules from support vector machines. Fuzzy Sets Syst 158:2057–2077MathSciNetMATHCrossRef
6.
Zurück zum Zitat Chen Z, Li J, Wei L (2007) A multiple kernel support vector machine scheme for feature selection and rule extraction from gene expression data of cancer tissue. Artif Intell Med 41:161–175CrossRef Chen Z, Li J, Wei L (2007) A multiple kernel support vector machine scheme for feature selection and rule extraction from gene expression data of cancer tissue. Artif Intell Med 41:161–175CrossRef
7.
Zurück zum Zitat Chiang J, Hao P (2004) Support vector learning mechanism for fuzzy rule-based modeling: a new approach. IEEE Trans Fuzzy Syst 12(1):1–12CrossRef Chiang J, Hao P (2004) Support vector learning mechanism for fuzzy rule-based modeling: a new approach. IEEE Trans Fuzzy Syst 12(1):1–12CrossRef
8.
Zurück zum Zitat Crammer K, Singer Y (2000) On the learnability and design of output codes for multiclass problems. In: COLT’00 proceedings of the thirteenth annual conference on computational learning theory, pp 35–46 Crammer K, Singer Y (2000) On the learnability and design of output codes for multiclass problems. In: COLT’00 proceedings of the thirteenth annual conference on computational learning theory, pp 35–46
9.
Zurück zum Zitat Cristianini N, Shawe-Taylor J (2000) An introduction to support vector machines and other kernel—based learning methods. Cambridge University Press, Cambridge, MACrossRef Cristianini N, Shawe-Taylor J (2000) An introduction to support vector machines and other kernel—based learning methods. Cambridge University Press, Cambridge, MACrossRef
10.
Zurück zum Zitat Diederich J (ed) (2008) Rule extraction from support vector machines, studies in computational intelligence, vol 80. Springer, Berlin, pp 3–30CrossRef Diederich J (ed) (2008) Rule extraction from support vector machines, studies in computational intelligence, vol 80. Springer, Berlin, pp 3–30CrossRef
11.
Zurück zum Zitat Fu X, Ong CJ, Keerthi S, Hung GG, Goh L (2004) Extracting the knowledge embedded in support vector machines, international joint conference on neural networks (IJCNN’04), Budapest, July 25–29, 2004, CDROM Fu X, Ong CJ, Keerthi S, Hung GG, Goh L (2004) Extracting the knowledge embedded in support vector machines, international joint conference on neural networks (IJCNN’04), Budapest, July 25–29, 2004, CDROM
12.
Zurück zum Zitat Fung G, Sandilya S, Rao R (2005) Rule extraction from linear support vector machines. In: Proceedings of the eleventh SIGKDD international conference on knowledge discovery and data mining Fung G, Sandilya S, Rao R (2005) Rule extraction from linear support vector machines. In: Proceedings of the eleventh SIGKDD international conference on knowledge discovery and data mining
13.
Zurück zum Zitat Gonçalves LB, Vellasco MMBR, Pacheco MAC, de Souza FJ (2006) Inverted hierarchical neuro—fuzzy BSP system: a novel neuro—fuzzy model for pattern classification and rule extraction in databases. IEEE Transact Syst Man Cybern Part C Appl Rev 36–2:236–248CrossRef Gonçalves LB, Vellasco MMBR, Pacheco MAC, de Souza FJ (2006) Inverted hierarchical neuro—fuzzy BSP system: a novel neuro—fuzzy model for pattern classification and rule extraction in databases. IEEE Transact Syst Man Cybern Part C Appl Rev 36–2:236–248CrossRef
14.
Zurück zum Zitat Haykin S (1999) Neural networks—a comprehensive foundation. Macmillan College Publishing Company, New YorkMATH Haykin S (1999) Neural networks—a comprehensive foundation. Macmillan College Publishing Company, New YorkMATH
15.
Zurück zum Zitat Heiler M, Cremers D, Schnörr C (2001) Efficient feature subset selection for support vector machines, technical report 21/2001, computer sciences series, University of Mannheim, Germany Heiler M, Cremers D, Schnörr C (2001) Efficient feature subset selection for support vector machines, technical report 21/2001, computer sciences series, University of Mannheim, Germany
16.
Zurück zum Zitat Hsu C-W, Lin C-J (2002) A comparison on methods for multi-class support vector machines. IEEE Transact Neural Netw 13–2:415–425 Hsu C-W, Lin C-J (2002) A comparison on methods for multi-class support vector machines. IEEE Transact Neural Netw 13–2:415–425
17.
Zurück zum Zitat John G, Kohavi R, Pfleger K (1994) Irrelevant features and the subset selection problem. In: Proceedings of the 11th international conference on machine learning, pp 121–129 John G, Kohavi R, Pfleger K (1994) Irrelevant features and the subset selection problem. In: Proceedings of the 11th international conference on machine learning, pp 121–129
18.
Zurück zum Zitat Fayyad UM, Piatetsky-Shapiro G, Smyth P, Uthurusamy R (1996) Advances in knowledge discovery and data mining. MIT Press, Cambridge Fayyad UM, Piatetsky-Shapiro G, Smyth P, Uthurusamy R (1996) Advances in knowledge discovery and data mining. MIT Press, Cambridge
19.
Zurück zum Zitat Kressel UH-G (1999) Pairwise classification and support vector machines. In: Schölkopf B, Burges CJC, Smola AJ (eds) Advances in kernel methods: support vector learning. MIT Press, Cambridge, MA, pp 225–268 Kressel UH-G (1999) Pairwise classification and support vector machines. In: Schölkopf B, Burges CJC, Smola AJ (eds) Advances in kernel methods: support vector learning. MIT Press, Cambridge, MA, pp 225–268
20.
Zurück zum Zitat Lanas AI (2000) Sistemas neuro—fuzzy hierárquicos BSP para Previsão e Extração de Regras Fuzzy em Aplicações de Mineração de Dados, M.S. thesis, Department Electrical Engineering, PUC-Rio (in Portuguese) Lanas AI (2000) Sistemas neuro—fuzzy hierárquicos BSP para Previsão e Extração de Regras Fuzzy em Aplicações de Mineração de Dados, M.S. thesis, Department Electrical Engineering, PUC-Rio (in Portuguese)
21.
Zurück zum Zitat Leng G, McGinnity TM (2004) G. Prasad, an approach for on-line extraction of fuzzy rules using a self-organising fuzzy neural network, in: fuzzy sets system 150:211–243MathSciNet Leng G, McGinnity TM (2004) G. Prasad, an approach for on-line extraction of fuzzy rules using a self-organising fuzzy neural network, in: fuzzy sets system 150:211–243MathSciNet
22.
Zurück zum Zitat Mendel JM (1995) Fuzzy logic systems for engineering: a tutorial. Proceed IEEE 83–3:345–377CrossRef Mendel JM (1995) Fuzzy logic systems for engineering: a tutorial. Proceed IEEE 83–3:345–377CrossRef
23.
Zurück zum Zitat Moody J, Utans J (1992) Principled architecture selection for neural networks: application to corporate bond rating prediction, advances in neural information processing systems 4. Morgan Kaufmann Publishers, Los Altos, CA, pp 683–690 Moody J, Utans J (1992) Principled architecture selection for neural networks: application to corporate bond rating prediction, advances in neural information processing systems 4. Morgan Kaufmann Publishers, Los Altos, CA, pp 683–690
24.
Zurück zum Zitat Núñez H, Angulo C, Català A (2002) Rule extraction from support vectors machines, European symposium on artificial neural networks (ESANN), pp 107–112 Núñez H, Angulo C, Català A (2002) Rule extraction from support vectors machines, European symposium on artificial neural networks (ESANN), pp 107–112
25.
Zurück zum Zitat Núñez H, Angulo C, Catala A (2006) Rule-based learning systems for support vector machines. Neural Process Lett 24:1–18CrossRef Núñez H, Angulo C, Catala A (2006) Rule-based learning systems for support vector machines. Neural Process Lett 24:1–18CrossRef
26.
Zurück zum Zitat Pitiraggon P, Benjathepanun N, Banditvilai S, Boonjing V (2010) Fuzzy rules generation and extraction from support vector machine based on kernel function firing signals. Int J Eng Appl Sci 6(4):244–251 Pitiraggon P, Benjathepanun N, Banditvilai S, Boonjing V (2010) Fuzzy rules generation and extraction from support vector machine based on kernel function firing signals. Int J Eng Appl Sci 6(4):244–251
27.
Zurück zum Zitat Rifkin R, Klautau A (2004) In defense of one-vs-all classification. J Mach Leran Res 5:101–141MathSciNetMATH Rifkin R, Klautau A (2004) In defense of one-vs-all classification. J Mach Leran Res 5:101–141MathSciNetMATH
28.
Zurück zum Zitat Sculley D, Wachman G (2007) Relaxed online SVMs for spam filtering, In: the thirtieth annual ACM SIGIR conference proceedings Sculley D, Wachman G (2007) Relaxed online SVMs for spam filtering, In: the thirtieth annual ACM SIGIR conference proceedings
29.
Zurück zum Zitat Vapnik VN (1998) Statistical learning theory. Wiley, New YorkMATH Vapnik VN (1998) Statistical learning theory. Wiley, New YorkMATH
30.
Zurück zum Zitat Vapnik VN, Golowich SE, Smola A (1997) Support vector method for function approximation, regression estimation, and signal processing, In: advances in neural information processing system 9, Morgan Kaufmann, pp 281–287 Vapnik VN, Golowich SE, Smola A (1997) Support vector method for function approximation, regression estimation, and signal processing, In: advances in neural information processing system 9, Morgan Kaufmann, pp 281–287
31.
Zurück zum Zitat Wang L-X, Mendel JM (1992) Generating fuzzy rules by learning from examples. IEEE Transact Syst Man Cybern 22–6:1414–1427MathSciNetCrossRef Wang L-X, Mendel JM (1992) Generating fuzzy rules by learning from examples. IEEE Transact Syst Man Cybern 22–6:1414–1427MathSciNetCrossRef
32.
Zurück zum Zitat Weston J, Watkins C (1998) Multi-class support vector machines, technical report CSD-TR-98-04, Royal Holloway Weston J, Watkins C (1998) Multi-class support vector machines, technical report CSD-TR-98-04, Royal Holloway
33.
Zurück zum Zitat Zhang J, Jin R, Yang YM, Hauptmann A (2003) A modified logistic regression: an approximation to SVM and its applications in large-scale text categorization, In: Proceedings of the twentieth international conference, Washington, DC, USA, August 21–24, 2003, pp 888–895 Zhang J, Jin R, Yang YM, Hauptmann A (2003) A modified logistic regression: an approximation to SVM and its applications in large-scale text categorization, In: Proceedings of the twentieth international conference, Washington, DC, USA, August 21–24, 2003, pp 888–895
Metadaten
Titel
Fuzzy rules extraction from support vector machines for multi-class classification
verfasst von
Adriana da Costa F. Chaves
Marley Maria B. R. Vellasco
Ricardo Tanscheit
Publikationsdatum
01.06.2013
Verlag
Springer-Verlag
Erschienen in
Neural Computing and Applications / Ausgabe 7-8/2013
Print ISSN: 0941-0643
Elektronische ISSN: 1433-3058
DOI
https://doi.org/10.1007/s00521-012-1048-5

Weitere Artikel der Ausgabe 7-8/2013

Neural Computing and Applications 7-8/2013 Zur Ausgabe