nach oben

Cognitive Computation

Erschienen in:

01.09.2014

A Learner-Independent Knowledge Transfer Approach to Multi-task Learning

verfasst von: Shaoning Pang, Fan Liu, Youki Kadobayashi, Tao Ban, Daisuke Inoue

Erschienen in: Cognitive Computation | Ausgabe 3/2014

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

This paper proposes a learner-independent multi-task learning (MTL) scheme in which knowledge transfer (KT) is running beyond the learner. In the proposed KT approach, we use minimum enclosing balls (MEBs) as knowledge carriers to extract and transfer knowledge from one task to another. Since the knowledge presented in MEB can be decomposed as raw data, it can be incorporated into any learner as additional training data for a new learning task to improve the learning rate. The effectiveness and robustness of the proposed KT is evaluated, respectively, on multi-task pattern recognition problems derived from synthetic datasets, UCI datasets, and real face image datasets, using classifiers from different disciplines for MTL. The experimental results show that multi-task learners using KT via MEB carriers perform better than learners without-KT, and this has been successfully applied to different classifiers such as k nearest neighbor and support vector machines.

Vorheriger Artikel An Improved Fault-Tolerant Objective Function and Learning Algorithm for Training the Radial Basis Function Neural Network

Nächster Artikel Abductive Learning Ensembles for Hand Shape Identification

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

McClelland J. Is a machine realization of truly human-like intelligence achievable? Cogn Comput. 2009;1(1):17–21.

Taylor JG. Cognitive computation. Cogn Comput. 2009;1(1):4–16.

Pang S, Ozawa S, Kasabov N. Incremental linear discriminant analysis for classification of data streams. IEEE Trans Neural Netw. 2005;35(5):905–914.

Pang S, Zhu L, Chen G, Sarrafzadeh A, Ban T, Inoue D. Dynamic class imbalance learning for incremental LPSVM. Neural Netw. 2013;44(2013):87–100.

Kumar P, Mitchell JSB, Yildirim EA. Approximate minimum enclosing balls in high dimensions using core-sets. J Exp Algorithm (JEA). 2003;8:11.

Chan TM. Approximating the diameter, width, smallest enclosing cylinder, and minimum-width annulus. In: Proceedings of the sixteenth annual symposium on computational geometry. Clear Water Bay, Kowloon, Hong Kong: ACM New York, NY, USA, 2000;300–309.

Pan SJ, Yang Q. A survey on transfer learning. Knowl Data Eng IEEE Trans. 2009;(99):1–1.

Argyriou A, Maurer A, Pontil M. An algorithm for transfer learning in a heterogeneous environment. In: proceedings of the 2008 European conference on machine learning and knowledge discovery in databases—Part I. Antwerp, Belgium: Springer; 2008. p. 71–85. http://dl.acm.org/citation.cfm?id=1431959.

Lawrence N. Platt J. Learning to learn with the informative vector machine. In: The 21st international conference on machine learning. 2004.

10.

Knowledge Transfer via Multiple Model Local Structure Mapping, ser. Research papers, no. 978-1-60558-193-4. ACM, November 2008.

11.

Mitchell TM. The need for biases in learning generalizations. Tech Rep.; 1980.

12.

Ozawa S, Roy A, Roussinov D. A multitask learning model for online pattern recognition. IEEE Trans Neural Netw. 2009;20(3):430–445.PubMedCrossRef

13.

Yu K, Schwaighofer A, Tresp V, Ma W-Y, Zhang H. Collaborative ensemble learning: combining collaborative and content-based information filtering via hierarchical bayes. In: Proceedings of the 19th conference on uncertainty in artificial intelligence. 2003.

14.

Silver D, Mercer R. Selective functional transfer: inductive bias from related tasks. In: Proceedings of the IASTED international conference on artificial intelligence and soft computing (ASC2001), p. 182–189.

15.

Mitchell TM. Machine learning. New York; McGraw-Hill. 1997.

16.

Silver DL, McCracken P. Selective transfer of task knowledge using stochastic noise. In: Xiang IY, Chaib-draa B, editors. Advances in artificial intelligence, proceedings of the 16th conference of the Canadian society for computational studies of intelligence (AI’2003). Berlin: Springer. 2003;190–205.

17.

Ghosn J, Bengio Y. Bias learning, knowledge sharing. IEEE Trans Neural Netw. 2003;14(4):748–765.PubMedCrossRef

18.

Silver DL, Poirier R. Sequential consolidation of learned task knowledge. 2004.

19.

Silver DL. Selective transfer of neural network task knowledge. PhD Thesis, University of Western Ontario. 2000.

20.

Caruana R. Multitask learning. Machine Learn. 1997;28:41–75.CrossRef

21.

Jonathan B. Learning internal representations. In: Proceedings of the eighth international conference on computational learning theory. 1995.

22.

Intrator N, Edelman S. Making a low-dimensional representation suitable for diverse tasks. Norwell, MA, USA: Kluwer Academic. 1998.

23.

Thrun S. Is learning the n-th thing any easier than learning the first?. In: Advances in neural information processing systems. Cambridge: The MIT Press. 1996;640–646.

24.

Roy A, Kim LS, Mukhopadhyay S. A polynomial time algorithm for the construction and training of a class of multilayer perceptrons. Neural Netw. 1993;6(4):535–545.CrossRef

25.

Roy A, Govil S, Miranda R. An algorithm to generate radial basis function (rbf)-like nets for classification problems. Neural Netw. 1995;8(2):179–202.CrossRef

26.

Roy A, Mukhopadhyay S. Iterative generation of higher-order nets in polynomial time using linear programming. IEEE Trans Neural Netw. 1997;8(2):402–412.PubMedCrossRef

27.

Roy A, Govil S, Miranda R. A neural network learning theory and a polynomial time rbf algorithm. IEEE Trans Neural Netw. 1997;8(6):1301–1313.PubMedCrossRef

28.

Xue Y, Liao X, Carin L. Multi-task learning for classification with dirichlet process priors. J Mach Learn Res Research. 2007;8:35–63.

29.

Ferguson T. A bayesian analysis of some nonparametric problems. Ann Stat. 1973;1:209–230.CrossRef

30.

Good I. Some history of the hierarchical Bayesian methodology, ser. Bayesian statistics. Valencia, Valencia University Press. 1980

31.

Evgeniou T, Pontil M. Regularized multi-task learning. In: Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining. ACM. 2004;109–117.

32.

Ben-David S, Schuller R. Exploiting task relatedness for multiple task learning. In: Proceedings of computational learning theory (COLT). 2003.

33.

Eaton E, desJardins M. Knowledge transfer with a multiresolution ensemble of classifiers. In: ICML-06 workshop on structural knowledge transfer for machine learning, June 29, Pittsburgh, PA. 2006.

34.

Eaton E. Multi-resolution learning for knowledge transfer. Boston, MA; AAAI Press. July 16–20 2006, [Doctoral Consortium].

35.

Eaton E, desJardins M, Stevenson J. Using multiresolution learning for transfer in image classification. In: aaai07. AAAI Press. 2007.

36.

Megiddo N. Linear-time algorithms for linear programming in r ³ and related problems. SIAM J Comput. 1983;12(4):759–776.CrossRef

37.

Welzl E. Smallest enclosing disks (balls and ellipsoids). Results New Trends Comput Sci. 1991;359–370.

38.

Badoiu M. Optimal core sets for balls. In: In DIMACS workshop on computational geometry. 2002.

39.

Chapelle O, Scholkopf B, Zien A. Semi-supervised learning. Cambridge, MA: MIT Press. 2006.CrossRef

40.

Bache K, Lichman M. UCI machine learning repository,” 2013. [Online]. http://archive.ics.uci.edu/ml.

41.

Kim M-S, Kim D, Lee S-Y. Face recognition descriptor using the embedded hmm with the 2nd-order block-specific eigenvectors. ISO/IEC JTC1/SC21/WG11/M7997, Jeju, 2002.

42.

Ozawa S, Pang S, Kasabov N. Online feature selection for adaptive evolving connectionist systems. Int J Innov Comput Inf Control. 2006;2(1):181–192.

43.

Ozawa S, Pang S, Kasabov N. Incremental learning of chunk data for on-line pattern classification systems. IEEE Trans Neural Netw. 2008;19(6):1061–1074.

44.

Ozawa S, Pang S, Kasabov N. Incremental learning of feature space and classifier for on-line pattern recognition. Int J Knowl Based Intell Eng Syst. 2006;10:57–65.

Titel: A Learner-Independent Knowledge Transfer Approach to Multi-task Learning
verfasst von: Shaoning Pang
Fan Liu
Youki Kadobayashi
Tao Ban
Daisuke Inoue
Publikationsdatum: 01.09.2014
Verlag: Springer US
Erschienen in: Cognitive Computation / Ausgabe 3/2014
Print ISSN: 1866-9956
Elektronische ISSN: 1866-9964
DOI: https://doi.org/10.1007/s12559-013-9238-8

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Weitere Artikel der Ausgabe 3/2014

A Gradient-Based Neural Network Method for Solving Strictly Convex Quadratic Programming Problems

An Insight into Extreme Learning Machines: Random Neurons, Random Features and Kernels

Modular Composite Representation

Abductive Learning Ensembles for Hand Shape Identification

A Voting Optimized Strategy Based on ELM for Improving Classification of Motor Imagery BCI Data

Modelling Task-Dependent Eye Guidance to Objects in Pictures