Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Published in:
BeatLex: Summarizing and Forecasting Time Series with Patterns Read chapter Read first chapter

2017 | OriginalPaper | Chapter

Lifelong Learning with Gaussian Processes

Authors : Christopher Clingerman, Eric Eaton

Published in: Machine Learning and Knowledge Discovery in Databases

Publisher: Springer International Publishing

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Recent developments in lifelong machine learning have demonstrated that it is possible to learn multiple tasks consecutively, transferring knowledge between those tasks to accelerate learning and improve performance. However, these methods are limited to using linear parametric base learners, substantially restricting the predictive power of the resulting models. We present a lifelong learning algorithm that can support non-parametric models, focusing on Gaussian processes. To enable efficient online transfer between Gaussian process models, our approach assumes a factorized formulation of the covariance functions, and incrementally learns a shared sparse basis for the models’ parameterizations. We show that this lifelong learning approach is highly computationally efficient, and outperforms existing methods on a variety of data sets.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
previous chapter Learning Task Clusters via Sparsity Grouped Multitask Learning
next chapter Personalized Tag Recommendation for Images Using Deep Transfer Learning
Appendix
Available only for authorised users
Literature
1.
Álvarez, M.A., Lawrence, N.D.: Computationally efficient convolved multiple output Gaussian processes. J. Mach. Learn. Res. 12, 1459–1500 (2011)MathSciNetMATH
2.
Álvarez, M.A., Luengo, D., Titsias, M.K., Lawrence, N.D.: Efficient multioutput Gaussian processes through variational inducing kernels. In: Proceedings of the 13th International Conference on Artificial Intelligence and Statistics (AISTATS), pp. 25–32 (2010)
3.
Bou Ammar, H., Eaton, E., Ruvolo, P., Taylor, M.E.: Online multi-task learning for policy gradient methods. In: Proceedings of the 31st International Conference on Machine Learning (ICML), June 2014
4.
Bonilla, E.V., Chai, K.M., Williams, C.: Multi-task Gaussian process prediction. In: Advances in Neural Information Processing Systems (NIPS), pp. 153–160 (2008)
5.
Bou Ammar, H., Eaton, E., Luna, J.M., Ruvolo, P.: Autonomous cross-domain knowledge transfer in lifelong policy gradient reinforcement learning. In: Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), July 2015
6.
Bou Ammar, H., Tutunov, R., Eaton, E.: Safe policy search for lifelong reinforcement learning with sublinear regret. In: Proceedings of the 32nd International Conference on Machine Learning (ICML), July 2015
7.
Chalupka, K., Williams, C.K., Murray, I.: A framework for evaluating approximation methods for Gaussian process regression. J. Mach. Learn. Res. 14(1), 333–350 (2013)MathSciNetMATH
8.
Chen, Z., Liu, B.: Lifelong Machine Learning. Synthesis Lectures on Artificial Intelligence and Machine Learning. Morgan & Claypool Publishers, San Rafael (2016)
9.
Deisenroth, M., Ng, J.W.: Distributed Gaussian processes. In: Proceedings of the 32nd International Conference on Machine Learning (ICML), pp. 1481–1490 (2015)
10.
Fei, G., Wang, S., Liu, B.: Learning cumulatively to become more knowledgeable. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2016)
11.
12.
Isele, D., Luna, J.M., Eaton, E., Gabriel, V., Irwin, J., Kallaher, B., Taylor, M.E.: Lifelong learning for disturbance rejection on mobile robots. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2016)
13.
Kamar, E., Kapoor, A., Horvitz, E.: Lifelong learning for acquiring the wisdom of the crowd. In: Proceedings of the 23rd International Joint Conference on Artificial Intelligence (IJCAI) (2013)
14.
Kumar, A., Daumé III, H.: Learning task grouping and overlap in multi-task learning. In: Proceedings of the 29th International Conference on Machine Learning (ICML) (2012)
15.
Maurer, A., Pontil, M., Romera-Paredes, B.: Sparse coding for multitask and transfer learning. In: Proceedings of the 30th International Conference on Machine Learning (ICML), pp. 343–351, May 2013
16.
Mitchell, T.M., Cohen, W.W., Talukdar, P.P., Betteridge, J., Carlson, A., Gardner, M., Kisiel, B., Krishnamurthy, J., et al.: Never ending learning. In: Proceedings of the 29th AAAI Conference on Artificial Intelligence (2015)
17.
Nguyen-Tuong, D., Peters, J.R., Seeger, M.: Local Gaussian process regression for real time online model learning. In: Advances in Neural Information Processing Systems (NIPS), pp. 1193–1200 (2009)
18.
Park, S., Choi, S.: Hierarchical Gaussian process regression. In: Proceedings of the 2nd Asian Conference on Machine Learning (ACML), pp. 95–110 (2010)
19.
Pentina, A., Urner, R.: Lifelong learning with weighted majority votes. In: Advances in Neural Information Processing Systems (NIPS), pp. 3612–3620 (2016)
20.
Quinonero-Candela, J., Rasmussen, C.E.: A unifying view of sparse approximate Gaussian process regression. J. Mach. Learn. Res. 6, 1939–1959 (2005)MathSciNetMATH
21.
Rakitsch, B., Lippert, C., Borgwardt, K., Stegle, O.: It is all in the noise: efficient multi-task Gaussian process inference with structured residuals. In: Advances in Neural Information Processing Systems (NIPS), pp. 1466–1474 (2013)
22.
Rasmussen, C.E., Williams, C.K.I.: Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning). The MIT Press, Cambridge (2005)
23.
Ruvolo, P., Eaton, E.: ELLA: an efficient lifelong learning algorithm. In: Proceedings of the 30th International Conference on Machine Learning (ICML), June 2013
24.
Snelson, E., Ghahramani, Z.: Sparse Gaussian processes using pseudo-inputs. In: Advances in Neural Information Processing Systems (NIPS), pp. 1257–1264 (2005)
25.
Thrun, S.: A lifelong learning perspective for mobile robot control. In: Proceedings of the IEEE/RSJ/GI International Conference on Intelligent Robots and Systems (IROS) ‘Advanced Robotic Systems and the Real World’ (1994)
26.
Tsanas, A., Little, M.A., McSharry, P.E., Ramig, L.O.: Accurate telemonitoring of Parkinson’s disease progression by noninvasive speech tests. IEEE Trans. Biomed. Eng. 57(4), 884–893 (2010)CrossRef
27.
Wang, B., Pineau, J.: Generalized dictionary for multitask learning with boosting. In: Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI) (2016)
28.
29.
Yu, K., Tresp, V., Schwaighofer, A.: Learning Gaussian processes from multiple tasks. In: Proceedings of the 22nd International Conference on Machine Learning (ICML), pp. 1012–1019 (2005)
Metadata
Title
Lifelong Learning with Gaussian Processes
Authors
Christopher Clingerman
Eric Eaton
Copyright Year
2017
Book
Machine Learning and Knowledge Discovery in Databases

Print ISBN: 978-3-319-71245-1

Electronic ISBN: 978-3-319-71246-8

Copyright Year: 2017

DOI
https://doi.org/10.1007/978-3-319-71246-8_42

Premium Partner

    Image Credits