Skip to main content
Top

2016 | OriginalPaper | Chapter

Negative Correlation Learning with Difference Learning

Author : Yong Liu

Published in: Computational Intelligence and Intelligent Systems

Publisher: Springer Singapore

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

In order to learn a given data set, a learning system often has to learn too much on some data points in the given data set in order to learn well the rest of the given data. Such unnecessary learning might lead to both the higher complexity and overfitting in the learning system. In order to control the complexity of neural network ensembles, difference learning is introduced into negative correlation learning. The idea of difference learning is to let each individual in an ensemble learn to be different to the ensemble on some selected data points when the outputs of the ensemble are too close to the target values of these data points. It has been found that such difference learning could control not only overfitting in an ensemble, but also weakness among the individuals in the ensemble. Experimental results were conducted to show how such difference learning could create rather weak learners in negative correlation learning.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Fahlman, S.E., Lebiere, C.: The cascade-correlation learning architecture. In: Touretzky, D.S. (ed.) Advances in Neural Information Processing Systems 2, pp. 524–532. Morgan Kaufmann, San Mateo, CA (1990) Fahlman, S.E., Lebiere, C.: The cascade-correlation learning architecture. In: Touretzky, D.S. (ed.) Advances in Neural Information Processing Systems 2, pp. 524–532. Morgan Kaufmann, San Mateo, CA (1990)
2.
go back to reference Śmieja, F.J.: Neural network constructive algorithms: trading generalization for learning efficiency? Circuits Syst. Sig. Process. 12(2), 331–374 (1993)CrossRefMATH Śmieja, F.J.: Neural network constructive algorithms: trading generalization for learning efficiency? Circuits Syst. Sig. Process. 12(2), 331–374 (1993)CrossRefMATH
3.
go back to reference Kwok, T.-Y., Yeung, D.-Y.: Constructive algorithms for structure learning in feedforward neural networks for regression problems. IEEE Trans. Neural Netw. 8(3), 630–645 (1997)CrossRef Kwok, T.-Y., Yeung, D.-Y.: Constructive algorithms for structure learning in feedforward neural networks for regression problems. IEEE Trans. Neural Netw. 8(3), 630–645 (1997)CrossRef
4.
go back to reference Mozer, M.C., Smolensky, P.: Skeletonization: a technique for trimming the fat from a network via relevance assessment. Connect. Sci. 1, 3–26 (1989)CrossRef Mozer, M.C., Smolensky, P.: Skeletonization: a technique for trimming the fat from a network via relevance assessment. Connect. Sci. 1, 3–26 (1989)CrossRef
5.
go back to reference Sietsma, J., Dow, R.J.F.: Creating artificial neural networks that generalize. Neural Netw. 4, 67–79 (1991)CrossRef Sietsma, J., Dow, R.J.F.: Creating artificial neural networks that generalize. Neural Netw. 4, 67–79 (1991)CrossRef
6.
go back to reference LeCun, Y., Denker, J.S., Solla, S.A.: Optimal brain damage. In: Touretzky, D.S. (ed.) Advances in Neural Information Processing Systems 2, pp. 598–605. Morgan Kaufmann, San Mateo, CA (1990) LeCun, Y., Denker, J.S., Solla, S.A.: Optimal brain damage. In: Touretzky, D.S. (ed.) Advances in Neural Information Processing Systems 2, pp. 598–605. Morgan Kaufmann, San Mateo, CA (1990)
7.
go back to reference Hassibi, B., Stork, D.G.: Second derivatives for network pruning: optimal brain surgeon. In: Hanson, S.J., Cowan, J.D., Giles, C.L. (eds.) Advances in Neural Information Processing Systems 5, pp. 164–171. Morgan Kaufmann, San Mateo, CA (1993) Hassibi, B., Stork, D.G.: Second derivatives for network pruning: optimal brain surgeon. In: Hanson, S.J., Cowan, J.D., Giles, C.L. (eds.) Advances in Neural Information Processing Systems 5, pp. 164–171. Morgan Kaufmann, San Mateo, CA (1993)
8.
go back to reference Tolstrup, N.: Pruning of a large network by optimal brain damage and surgeon: an example from biological sequence analysis. Int. J. Neural Syst. 6(1), 31–42 (1995)CrossRef Tolstrup, N.: Pruning of a large network by optimal brain damage and surgeon: an example from biological sequence analysis. Int. J. Neural Syst. 6(1), 31–42 (1995)CrossRef
9.
go back to reference Schapire, R.E.: The strength of weak learnability. Mach. Learn. 5, 197–227 (1990) Schapire, R.E.: The strength of weak learnability. Mach. Learn. 5, 197–227 (1990)
10.
go back to reference Liu, Y., Yao, X.: Simultaneous training of negatively correlated neural networks in an ensemble. IEEE Trans. Syst. Man Cybern. Part B Cybern. 29(6), 716–725 (1999)CrossRef Liu, Y., Yao, X.: Simultaneous training of negatively correlated neural networks in an ensemble. IEEE Trans. Syst. Man Cybern. Part B Cybern. 29(6), 716–725 (1999)CrossRef
11.
go back to reference Liu, Y.: A balanced ensemble learning with adaptive error functions. In: Kang, L., Cai, Z., Yan, X., Liu, Y. (eds.) ISICA 2008. LNCS, vol. 5370, pp. 1–8. Springer, Heidelberg (2008)CrossRef Liu, Y.: A balanced ensemble learning with adaptive error functions. In: Kang, L., Cai, Z., Yan, X., Liu, Y. (eds.) ISICA 2008. LNCS, vol. 5370, pp. 1–8. Springer, Heidelberg (2008)CrossRef
12.
go back to reference Liu, Y.: Balanced learning for ensembles with small neural networks. In: Cai, Z., Li, Z., Kang, Z., Liu, Y. (eds.) ISICA 2009. LNCS, vol. 5821, pp. 163–170. Springer, Heidelberg (2009)CrossRef Liu, Y.: Balanced learning for ensembles with small neural networks. In: Cai, Z., Li, Z., Kang, Z., Liu, Y. (eds.) ISICA 2009. LNCS, vol. 5821, pp. 163–170. Springer, Heidelberg (2009)CrossRef
13.
go back to reference Liu, Y.: Create weak learners with small neural networks by balanced ensemble learning. In: Proceedings of the 2011 IEEE International Conference on Signal Processing, Communications and Computing (2011) Liu, Y.: Create weak learners with small neural networks by balanced ensemble learning. In: Proceedings of the 2011 IEEE International Conference on Signal Processing, Communications and Computing (2011)
14.
go back to reference Liu, Y.: Target shift awareness in balanced ensemble learning. In: Proceedings of the 3rd International Conference on Awareness Science and Technology Liu, Y.: Target shift awareness in balanced ensemble learning. In: Proceedings of the 3rd International Conference on Awareness Science and Technology
15.
go back to reference Liu, Y.: Balancing ensemble learning through error shift. In: Proceedings of the Fourth International Workshop on Advanced Computational Intelligence Liu, Y.: Balancing ensemble learning through error shift. In: Proceedings of the Fourth International Workshop on Advanced Computational Intelligence
Metadata
Title
Negative Correlation Learning with Difference Learning
Author
Yong Liu
Copyright Year
2016
Publisher
Springer Singapore
DOI
https://doi.org/10.1007/978-981-10-0356-1_27

Premium Partner