Skip to main content
Fachgebiete
Automobil + Motoren Bauwesen + Immobilien Business IT + Informatik Elektrotechnik + Elektronik Energie + Nachhaltigkeit Finance + Banking Management + Führung Marketing + Vertrieb Maschinenbau + Werkstoffe Versicherung + Risiko
DE
EN
Bücher
Zeitschriften
Themenseiten
Organisationspsychologie Projektmanagement Marketing Smart Manufacturing
Weitere Formate
Podcasts Webinare Technik Webinare Wirtschaft Kongresse Veranstaltungskalender Awards
Jetzt Einzelzugang starten
Zugang für Unternehmen
Referenzkunden
SLX-Digitalkonferenz: Zukunftswerkstatt 2024

Mehr Umsatz mit Top-Kontakten

Am 7. Mai erfahren Sie, wie Vertriebsteams Leadgenerierung, Leadnurturing und Leadstrategien effizient steuern können.
Erweiterte Suche
Anmelden
nach oben
Progress in Artificial Intelligence
Erschienen in: Progress in Artificial Intelligence 4/2012

01.12.2012 | Regular Paper

Information Theoretic Learning and local modeling for binary and multiclass classification

verfasst von: Iago Porto-Díaz , David Martínez-Rego, Amparo Alonso-Betanzos, Oscar Fontenla-Romero

Erschienen in: Progress in Artificial Intelligence | Ausgabe 4/2012

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

In this paper, a learning model for binary and multiclass classification based on local modeling and Information Theoretic Learning (ITL) is described. The training algorithm for the model works on two stages: first, a set of nodes are placed on the frontiers between classes using a modified clustering algorithm based on ITL. Each of these nodes defines a local model. Second, several one-layer neural networks, associated with these local models, are trained to locally classify the points in its proximity. The method is successfully applied to problems with a large amount of instances and high dimension like intrusion detection and microarray gene expression.
Vorheriger Artikel Binary relevance efficacy for multilabel classification
Nächster Artikel One iteration CHC algorithm for learning Bayesian networks: an effective and efficient algorithm for high dimensional problems

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Jetzt informieren
Literatur
1.
Alizadeh, A.A., Eisen, M.B., Davis, R.E., Ma, C., Lossos, I.S., Rosenwald, A., Boldrick, J.C., Sabet, H., Tran, T., Yu, X., et al.: Distinct types of diffuse large B-cell lymphoma identified by gene expression profiling. Nature 403(6769), 503–511 (2000)CrossRef
2.
Alon, U., Barkai, N., Notterman, D.A., Gish, K., Ybarra, S., Mack, D.: Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays. Proc. Natl. Acad. Sci. USA 96(12), 6745–6750 (1999)CrossRef
3.
Alonso-Betanzos, A., Sanchez-Marono, N., Carballal-Fortes, F.M., Suarez-Romero, J., Perez-Sanchez, B.: Classification of computer intrusions using fuctional networks: a comparative study. In: Proceedings of the ESANN, pp. 25–27 (2007)
4.
Bishop, C.M.: Neural networks for pattern recognition. Clarendon Press, Oxford (1995)
5.
Bolón-Canedo, V., Sánchez-Marono, N., Alonso-Betanzos, A.: On the effectiveness of discretization on gene selection of microarray data, pp. 3167–3174. In: Proceedings of International Joint Conference on Neural Networks, IJCNN (2010)
6.
Bolón-Canedo, V., Sanchez-Maroo, N., Alonso-Betanzos, A.: A combination of discretization and filter methods for improving classification performance in kdd cup 99 dataset. In: International Joint Conference on Neural Networks, IJCNN 2009, pp. 359–366. IEEE (2009)
7.
8.
9.
10.
Castillo, E., Fontenla-Romero, O., Guijarro-Berdiñas, B., Alonso-Betanzos, A.: A global optimum approach for one-layer neural networks. Neural Comput. 14(6), 1429–1449 (2002)MATHCrossRef
11.
Cover, T., Hart, P.: Nearest neighbor pattern classification. IEEE Trans. Inf. Theory 13(1), 21–27 (1967)MATHCrossRef
12.
13.
Cristianini, N., Shawe-Taylor, J.: An introduction to support vector machines and other kernel-based learning methods. Cambridge University Press, Cambridge (2000)
14.
Dasarathy, B.V., Sheela, B.V.: A composite classifier system design: concepts and methodology. Proc. IEEE 67(5), 708–713 (1979)CrossRef
15.
16.
Ding, C., Peng, H.: Minimum redundancy feature selection from microarray gene expression data. J. Bioinformatics Comput. Biol. 3(2), 185–206 (2005)MathSciNetCrossRef
17.
Duda, R.O., Hart, P.E., Stork, D.G.: Pattern classification, 2nd edn. Wiley, New York (2001)MATH
18.
Elkan, C.: Results of the kdd’99 classifier learning. ACM SIGKDD Explor. Newsl. 1(2), 63–64 (2000)CrossRef
19.
Fayyad, U., Irani, K.: Multi-interval discretization of continuous-valued attributes for classification, learning (1993)
20.
Fisher, R.A.: The use of multiple measurements in taxonomic problems. Annals Hum. Genet. 7(2), 179–188 (1936)
21.
Freund, Y., Schapire, R.E.: Experiments with a new boosting algorithm. In: International Workshop then Conference on Machine Learning, pp. 148–156. Morgan Kaufmann Publishers, Inc. (1996)
22.
Freund, Y., Schapire, R.E.: A decision-theoretic generalization of online learning and an application to boosting. J. Comput. System Sci. 55(1), 119–139 (1997)MathSciNetMATHCrossRef
23.
Fugate, M., Gattiker, J.R.: Computer intrusion detection with classification and anomaly detection, using SVMs. Int. J. Pattern Recognit. Artif. Intell. 17(3), 441–458 (2003)CrossRef
24.
Golub, T.R., Slonim, D.K., Tamayo, P., Huard, C., Gaasenbeek, M., Mesirov, J.P., Coller, H., Loh, M.L., Downing, J.R., Caligiuri, M.A.: Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. Science 286(5439), 531–537 (1999)CrossRef
25.
Gordon, G.J., Jensen, R.V., Hsiao, L.L., Gullans, S.R., Blumenstock, J.E., Ramaswamy, S., Richards, W.G., Sugarbaker, D.J., Bueno, R.: Translation of microarray data into clinically relevant cancer diagnostic tests using gene expression ratios in lung cancer and mesothelioma. Cancer Res. 62(17), 4963–4971 (2002)
26.
Hall, M.A.: Correlation-based feature selection for machine learning. PhD thesis, University of Waikato, Hamilton, New Zealand (1999)
27.
Kearns, M.J.: Thoughts on hypothesis boosting. ML Class Project 319, 320 (1988)
28.
Kiang, M.Y.: A comparative assessment of classification methods. Decis. Support Systems 35(4), 441–454 (2003)MathSciNetCrossRef
29.
Kuncheva, L.I.: Clustering-and-selection model for classifier combination. In: Proceedings of the Fourth International Conference on Knowledge-Based Intelligent Engineering Systems and Allied Technologies, 2000, vol. 1, pp. 185–188. IEEE (2000)
30.
Kuncheva, L.I.: Combining pattern classifiers: methods and algorithms (2004)
31.
Lehn-Schiøler, T., Hegde, A., Erdogmus, D., Principe, J.C.: Vector quantization using information theoretic concepts. Nat. Comput. 4(1), 39–51 (2005)MathSciNetCrossRef
32.
Levin, I.: Kdd-99 classifier learning contest: Llsoft’s results overview. SIGKDD Explor. 1(2), 67–75 (2000)CrossRef
33.
34.
Liu, H., Setiono, R.: Chi2: feature selection and discretization of numeric attributes. In: Proceedings of the Seventh IEEE International Conference on Tools with Artificial Intelligence, November 5–8, 1995, pp. 388–391. IEEE Comput. Soc. (1995)
35.
Liu, R., Yuan, B.: Multiple classifiers combination by clustering and selection. Inf. Fusion 2(3), 163–168 (2001)CrossRef
36.
Martinez-Rego, D., Fontenla-Romero O., Porto-Diaz, I., Alonso-Betanzos, A.: A new supervised local modelling classifier based on information theory. In: International Joint Conference on Neural Networks, 2009. IJCNN 2009, pp. 2014–2020. IEEE (2009)
37.
Nock R., Nielsen F.: A real generalization of discrete adaboost. In: Proceeding of the 2006 Conference on ECAI 2006: 17th European Conference on Artificial Intelligence, August 29–September 1, 2006, Riva del Garda, Italy, pp. 509–515. IOS Press (2006)
38.
Nutt, C.L., Mani, D.R., Betensky, R.A., Tamayo, P., Cairncross, J.G., Ladd, C., Pohl, U., Hartmann, C., McLaughlin, M.E., Batchelor, T.T.: Gene expression-based classification of malignant gliomas correlates better with survival than histological classification. Cancer Res. 63(7), 1602–1610 (2003)
39.
Petricoin III, E.F., Ardekani, A.M., Hitt, B.A., Levine, P.J., Fusaro, V.A., Steinberg, S.M., Mills, G.B., Simone, C., Fishman, D.A., Kohn, E.C., et al.: Use of proteomic patterns in serum to identify ovarian cancer. Lancet 359(9306), 572–577 (2002)CrossRef
40.
Pomeroy, S.L., Tamayo, P., Gaasenbeek, M., Sturla, L.M., Angelo, M., McLaughlin, M.E., Kim, J.Y.H., Goumnerova, L.C., Black, P.M., Lau, C., et al.: Prediction of central nervous system embryonal tumour outcome based on gene expression. Nature 415(6870), 436–442 (2002)CrossRef
41.
Porto-Díaz, I., Bolón-Canedo, V., Alonso-Betanzos, A., Fontenla-Romero, Ó.: Local modeling classifier for microarray gene-expression data. In: Artificial Neural Networks, ICANN 2010, pp. 11–20 (2010)
42.
Porto-Díaz, I., Bolón-Canedo, V., Alonso-Betanzos, A., Fontenla-Romero, O.: A study of performance on microarray data sets for a classifier based on information theoretic learning. Neural Netw. 24(8), 888–896 (2011)
43.
Porto-Díaz, I., Martínez-Rego, D., Alonso-Betanzos, A., Fontenla-Romero, O.: Combining feature selection and local modelling in the kdd cup 99 dataset. In: Artificial Neural Networks, ICANN 2009, pp. 824–833 (2009)
44.
Principe, J.C.: Information theoretic learning: Renyi’s entropy and kernel perspectives. Springer (2010)
45.
Principe, J.C., Xu, D., Zhao, Q., Fisher, J.W.: Learning from examples with information theoretic criteria. J. VLSI Signal Process. 26(1), 61–77 (2000)MATHCrossRef
46.
Quinlan, J.R.: Induction of decision trees. Mach. Learn. 1(1), 81–106 (1986)
47.
Quinlan, J.R.: C4. 5: programs for machine learning. Morgan kaufmann (1993)
48.
Rastrigin, L.A., Erenstein, R.H.: Method of collective recognition. Energoizdat, Moscow (1981)MATH
49.
Saeys, Y., Inza, I., Larrañaga, P.: A review of feature selection techniques in bioinformatics. Bioinformatics 23(19), 2507–2517 (2007)CrossRef
50.
Singh, D., Febbo, P.G., Ross, K., Jackson, D.G., Manola, J., Ladd, C., Tamayo, P., Renshaw, A.A., D’Amico, A.V., Richie, J.P., et al.: Gene expression correlates of clinical prostate cancer behavior. Cancer Cell 1(2), 203–209 (2002)
51.
Spira, A., Beane, J.E., Shah, V., Steiling, K., Liu, G., Schembri, F., Gilman, S., Dumas, Y.M., Calner, P., Sebastiani, P., et al.: Airway epithelial gene expression in the diagnostic evaluation of smokers with suspect lung cancer. Nature Med. 13(3), 361–366 (2007)CrossRef
52.
Tian, E., Zhan, F., Walker, R., Rasmussen, E., Ma, Y., Barlogie, B., Shaughnessy, Jr., J.D.: The role of the Wnt-signaling antagonist DKK1 in the development of osteolytic lesions in multiple myeloma. N. Engl. J. Med. 349(26), 2483–2494 (2003)
53.
van’t Veer, V., Laura, J., Dai, H., Van de Vijver, M.J., He, Y.D., Hart, A.A.M.: Gene expression profiling predicts clinical outcome of breast cancer. Nature 415(6871), 530–536 (2002)CrossRef
54.
55.
56.
Yang, Y., Webb, G.: Proportional k-interval discretization for naive-bayes classifiers. In: Machine Learning: ECML 2001, pp. 564–575 (2001)
57.
Zhao, Z., Liu, H.: Searching for interacting features. In: Proceedings of the 20th International Joint Conference on Artifical intelligence, pp. 1156–1161. Morgan Kaufmann Publishers Inc. (2007)
Metadaten
Titel
Information Theoretic Learning and local modeling for binary and multiclass classification
verfasst von
Iago Porto-Díaz
David Martínez-Rego
Amparo Alonso-Betanzos
Oscar Fontenla-Romero
Publikationsdatum
01.12.2012
Verlag
Springer-Verlag
Erschienen in
Progress in Artificial Intelligence / Ausgabe 4/2012
Print ISSN: 2192-6352
Elektronische ISSN: 2192-6360
DOI
https://doi.org/10.1007/s13748-012-0032-8

Weitere Artikel der Ausgabe 4/2012

Progress in Artificial Intelligence 4/2012 Zur Ausgabe

Regular Paper

Agreement technologies and their use in cloud computing environments

Regular Paper

Effective real-time visual object detection

Regular Paper

Binary relevance efficacy for multilabel classification

Regular Paper

Surrounding neighborhood-based SMOTE for learning from imbalanced data sets

Regular Paper

Extending the upper–lower edge detector by means of directional masks and OWA operators

Regular Paper

“I can tell you what it’s not”: active learning from counterexamples

Premium Partner

    Bildnachweise