Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Published in:
Prologue Read chapter Read first chapter

2012 | OriginalPaper | Chapter

7. Incremental Classifier Fusion and Its Applications in Industrial Monitoring and Diagnostics

Authors : Davy Sannen, Jean-Michel Papy, Steve Vandenplas, Edwin Lughofer, Hendrik Van Brussel

Published in: Learning in Non-Stationary Environments

Publisher: Springer New York

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Pattern recognition techniques have shown their usefulness for monitoring and diagnosing many industrial applications. The increasing production rates and the growing databases generated by these applications require learning techniques that can adapt their models incrementally, without revisiting previously used data. Ensembles of classifiers have been shown to improve the predictive accuracy as well as the robustness of classification systems. In this work, several well-known classifier fusion methods (Fuzzy Integral, Decision Templates, Dempster–Shafer Combination, and Discounted Dempster–Shafer Combination) are extended to allow incremental adaptation. Additionally, an incremental classifier fusion method using an evolving clustering approach is introduced—named Incremental Direct Cluster-based ensemble. A framework for strict incremental learning is proposed in which the ensemble and its member classifiers are adapted concurrently. The proposed incremental classifier fusion methods are evaluated within this framework for two industrial applications: online visual quality inspection of CD imprints and prediction of maintenance actions for copiers from a large historical database.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
previous chapter Making Early Predictions of the Accuracy of Machine Learning Classifiers
next chapter Instance-Based Classification and Regression on Data Streams
Literature
1.
Bey-Temsamani, A., Engels, M., Motten, A., Vandenplas, S., Ompusunggu, A.: A practical approach to combine data mining and prognostics for improved predictive maintenance. In: Proceedings of the Data Mining Case Studies Workshop, 15th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 37–44 (2009)
2.
3.
Breiman, L.: Pasting small votes for classification in large databases and on-line. Mach. Learn. 36(1–2), 85–103 (1999)CrossRef
4.
Breiman, L., Friedman, J., Olshen, R., Stone, C.: Classification and Regression Trees. Wadsworth International Group, Belmont, CA, USA (1984)MATH
5.
Cho, S., Kim, J.: Combining multiple neural networks by fuzzy integral for robust classification. IEEE Trans. Syst. Man Cybern. 25(2), 380–384 (1995)CrossRef
6.
7.
Dempster, A.: A generalization of Bayesian inference. J. Roy. Stat. Soc. B 30, 205–247 (1968)MathSciNet
8.
Diehl, C., Cauwenberghs, G.: SVM incremental learning, adaptation and optimization. In: Proceedings of the 2003 International Joint Conference on Neural Networks, pp. 2685–2690 (2003)
9.
Dietterich, T.: Ensemble methods in machine learning. In: J. Kittler, F. Roli (eds.) Multiple Classifier Systems, Lecture Notes in Computer Science, vol. 1857, pp. 1–15. Springer, Berlin/Heidelberg, Germany (2000)
10.
Duda, R., Hart, P.: Pattern Classification and Scene Analysis. John Wiley & Sons, New York (1973)MATH
11.
Duda, R., Hart, P., Stork, D.: Pattern Classification, 2nd edn. John Wiley & Sons, New York, USA (2000)
12.
Eitzinger, C., Gmainer, M., Heidl, W., Lughofer, E.: Increasing classification robustness with adaptive features. In: A. Gasteratos, M. Vincze, J. Tsotsos (eds.) Computer Vision Systems, Lecture Notes in Computer Science, vol. 5008, pp. 445–453. Springer, Berlin/Heidelberg (2008)CrossRef
13.
Feng, J.: An intelligent decision support system based on machine learning and dynamic track of psychological evaluation criterion. In: J. Kacpzryk (ed.) Intelligent Decision and Policy Making Support Systems, pp. 141–157. Springer, Berlin/Heidelberg (2008)CrossRef
14.
Filev, D., Tseng, F.: Novelty detection based machine health prognostics. In: Proceedings of the 2006 International Symposium on Evolving Fuzzy Systems, pp. 193–199 (2006)
15.
Freund, Y., Schapire, R.: A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. Syst. Sci. Int. 55(1), 119–139 (1997)MathSciNetMATHCrossRef
16.
Fung, G., Mangasarian, O.: Incremental support vector machine classification. In: Second SIAM International Conference on Data Mining, pp. 247–260 (2002)
17.
Gader, P., Mohamed, M., Keller, J.: Fusion of handwritten word classifiers. Pattern Recognit. Lett. 17(6), 577–584 (1996)CrossRef
18.
Gama, J.: Knowledge Discovery from Data Streams. Chapman & Hall / CRC, Boca Raton, Florida, USA (2010)MATHCrossRef
19.
20.
Hansen, L., Salamon, P.: Neural network ensembles. IEEE Trans. Pattern Anal. Mach. 12(10), 993–1001 (1990)CrossRef
21.
Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer Verlag, New York, USA (2001)MATH
22.
Huang, Y., Suen, C.: A method of combining multiple experts for the recognition of unconstrained handwritten numerals. IEEE Trans. Pattern Anal. Mach. 17(1), 90–94 (1995)CrossRef
23.
Jacobs, R., Jordan, M., Nowlan, S., Hinton, G.: Adaptive mixtures of local experts. Neural Comput. 3, 79–87 (1991)CrossRef
24.
Kerdprasop, N., Kerdprasop, K.: Data partitioning for incremental data mining. In: Proceedings of the First International Forum on Information and Computer Science, pp. 114–118. Hamamatsu, Japan (2003)
25.
Kittler, J., Hatef, M., Duin, R., Matas, J.: On combining classifiers. IEEE Trans. Pattern Anal. Mach. 20(3), 226–239 (1998)CrossRef
26.
Krogh, A., Vedelsby, J.: Neural network ensembles, cross validation, and active learning. In: G. Tesauro, D. Touretzky, T. Leen (eds.) Proceedings of the 7th Conference on Advances in Neural Information Processing Systems, vol. 7, pp. 231–238. MIT Press, Cambridge, MA, USA (1995)
27.
Kuncheva, L.: Using measures of similarity and inclusion for multiple classifier fusion by decision templates. Fuzzy Sets Syst. 122(3), 401–407 (2001)MathSciNetMATHCrossRef
28.
Kuncheva, L.: A theoretical study on six classifier fusion strategies. IEEE Trans. Pattern Anal. Mach. 24(2), 281–286 (2002)CrossRef
29.
Kuncheva, L.: Classifier ensembles for changing environments. In: F. Roli, J. Kittler, T. Windeatt (eds.) Multiple Classifier Systems, Lecture Notes in Computer Science, vol. 3077, pp. 1–15. Springer, Berlin/Heidelberg, Germany (2004)
30.
Kuncheva, L.: Combining Pattern Classifiers: Methods and Algorithms. Wiley, New Jersey (2004)MATHCrossRef
31.
Kuncheva, L., Bezdek, J., Duin, R.: Decision templates for multiple classifier fusion: An experimental comparison. Pattern Recognit. 34(2), 299–314 (2001)MATHCrossRef
32.
Kuncheva, L., Whitaker, C., Shipp, C., Duin, R.: Limits on the majority vote accuracy in classifier fusion. Pattern Anal. Appl. 6(1), 22–31 (2003)MathSciNetMATHCrossRef
33.
Li, W., Yue, H., Valle-Cervantes, S., Qin, S.: Recursive PCA for adaptive process monitoring. J. Process Control 10(5), 471–486 (2000)CrossRef
34.
Lim, C., Harrison, R.: Online pattern classification with multiple neural network systems: An experimental study. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 33, 235–247 (2003)
35.
Lughofer, E.: Extensions of vector quantization for incremental clustering. Pattern Recognition, 41(3), 995–1011 (2008)MATHCrossRef
36.
Lughofer, E.: Hybrid active learning for reducing the annotation efforts of operators in classification systems. Pattern Recognition, 45(2), 884–896 (2012)CrossRef
37.
Lughofer, E.: Evolving Fuzzy Systems: Methodologies, Advanced Concepts and Applications. Springer, Berlin/Heidelberg (2011)MATHCrossRef
38.
Lughofer, E., Guardiola, C.: On-line fault detection with data-driven evolving fuzzy models. J. Contr. Intell. Syst. 36(4), 307–317 (2008)MATH
39.
Lughofer, E., Smith, J., Caleb-Solly, P., Tahir, M., Eitzinger, C., Sannen, D., Nuttin, M.: Human-machine interaction issues in quality control based on online image classification. IEEE Trans. Syst. Man Cybern. Part A Syst. Humans 39(5), 960–971 (2009)CrossRef
40.
Nisbet, R., Elder, J., Miner, G.: Handbook of Statistical Analysis and Data Mining Applications. Academic Press, Burlington, MA, USA (2009)MATH
41.
Oza, N.: Online ensemble learning. Ph.D. thesis, University of California, USA (2001)
42.
Pang, S., Ozawa, S., Kasabov, N.: Incremental linear discriminant analysis for classification of data streams. IEEE Trans. Syst. Man Cybern. Part B Cybern. 35(5), 905–914 (2005)CrossRef
43.
Polikar, R., Upda, L., Upda, S., Honavar, V.: Learn++: An incremental learning algorithm for supervised neural networks. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 31(4), 497–508 (2001)CrossRef
44.
Polikar, R.: Ensemble based systems in decision making. IEEE Circuits Syst. Mag. 6(3), 21–45 (2006)CrossRef
45.
Quinlan, J.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Mateo, CA, USA (1993)
46.
Rogova, G.: Combining the results of several neural network classifiers. Neural Networks 7(5), 777–781 (1994)CrossRef
47.
Roubos, J., Setnes, M., Abonyi, J.: Learning fuzzy classification rules from data. Inform. Sciences 150, 77–93 (2003)MathSciNetCrossRef
48.
Rüping, S.: Incremental learning with support vector machines. Ph.D. thesis, Universität Dortmund (2002)
49.
Ruta, D., Gabrys, B.: An overview of classifier fusion methods. Comput. and Inform. Syst. J. 7(1), 1–10 (2000)
50.
Sannen, D.: A flexible framework for learning visual quality inspection: Information fusion approaches. Ph.D. thesis, Katholieke Universiteit Leuven, Belgium (2010)
51.
Sannen, D., Nuttin, M., Smith, J., Tahir, M., Caleb-Solly, P., Lughofer, E., Eitzinger, C.: An on-line interactive self-adaptive image classification framework. In: A. Gasteratos, M. Vincze, J. Tsotsos (eds.) Computer Vision Systems, Lecture Notes in Computer Science, vol. 5008, pp. 171–180. Springer, Berlin/Heidelberg, Germany (2008)
52.
Sannen, D., Van Brussel, H., Nuttin, M.: Classifier fusion using discounted Dempster–Shafer combination. In: P. Perner (ed.) Machine Learning and Data Mining in Pattern Recognition, Poster Proceedings, pp. 216–230. IBaI Publishing, Leipzig, Germany (2007)
53.
Shafer, G.: A Mathematical Theory of Evidence. Princeton University Press, Princeton, NJ, USA (1976)MATH
54.
Song, M., Wang, H.: Incremental estimation of gaussian mixture models for online data stream clustering. In: Proceedings of International Conference on Bioinformatics and its Applications (2004)
55.
Stone, M.: Cross-validatory choice and assessment of statistical predictions (with discussion). J. Roy. Stat. Soc. B 36, 111–147 (1974)MATH
56.
Sutton, R., Whitehead, S.: Online learning with random representations. In: Proceedings of the 10th International Conference on Machine Learning, pp. 314–321. Morgan Kaufmann, San Mateo, CA, USA (1993)
57.
Theodoridis, S., Koutroumbas, K.: Pattern Recognition, 4th edn. Academic Press, Burlington, MA, USA (2009)
58.
Utgoff, P.: Incremental induction of decision trees. Mach. Learn. 4, 161–186 (1989)CrossRef
59.
Valentini, G., Masulli, F.: Ensembles of learning machines. In: M. Marinaro, R. Tagliaferri (eds.) Neural Nets: 13th Italian Workshop on Neural Nets, Vietri sul Mare, Italy, 2002, Revised Papers, Lecture Notes in Computer Science, vol. 2486, pp. 3–22. Springer, Berlin/Heidelberg, Germany (2002)
60.
Vapnik, V., Golowich, S., Smola, A.: Support vector method for function approximation, regression estimation and signal processing. Adv. Neural Inf. Process. Syst. 9, 281–287 (1997)
61.
Wang, H., Fan, W., Yu, P., Han, J.: Mining concept-drifting data streams using ensemble classifiers. In: Proceedings of the 9th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 226–235. ACM, New York, NY, USA (2003)
62.
Wasserman, P.: Advanced Methods in Neural Computing. Van Nostrand Reinhold, New York, USA (1993)MATH
63.
Widmer, G., Kubat, M.: Learning in the presence of concept drift and hidden contexts. Mach. Learn. 23(1), 69–101 (1996)
64.
Wolpert, D., Macready, W.: No free lunch theorems for optimization. IEEE Trans. Evol. Comput. 1(1), 67–82 (1997)CrossRef
65.
Woods, K., Kegelmeyer, W., Bowyer, K.: Combination of multiple classifiers using local accuracy estimates. IEEE Trans. Pattern Anal. Mach. 19(4), 405–410 (1997)CrossRef
66.
Xu, L., Krzyźak, A., Suen, C.: Methods of combining multiple classifiers and their applications to handwriting recognition. IEEE Trans. Syst. Man Cybern. 22(3), 418–435 (1992)CrossRef
Metadata
Title
Incremental Classifier Fusion and Its Applications in Industrial Monitoring and Diagnostics
Authors
Davy Sannen
Jean-Michel Papy
Steve Vandenplas
Edwin Lughofer
Hendrik Van Brussel
Copyright Year
2012
Publisher
Springer New York
Book
Learning in Non-Stationary Environments

Print ISBN: 978-1-4419-8019-9

Electronic ISBN: 978-1-4419-8020-5

Copyright Year: 2012

DOI
https://doi.org/10.1007/978-1-4419-8020-5_7

Premium Partner

    Image Credits