Skip to main content

2020 | OriginalPaper | Buchkapitel

Data Association with Gaussian Processes

verfasst von : Markus Kaiser, Clemens Otte, Thomas A. Runkler, Carl Henrik Ek

Erschienen in: Machine Learning and Knowledge Discovery in Databases

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

The data association problem is concerned with separating data coming from different generating processes, for example when data comes from different data sources, contain significant noise, or exhibit multimodality. We present a fully Bayesian approach to this problem. Our model is capable of simultaneously solving the data association problem and the induced supervised learning problem. Underpinning our approach is the use of Gaussian process priors to encode the structure of both the data and the data associations. We present an efficient learning scheme based on doubly stochastic variational inference and discuss how it can be applied to deep Gaussian process priors.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Abadi, M., Agarwal, A., Barham, P., et al.: TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems (2015). tensorflow.org Abadi, M., Agarwal, A., Barham, P., et al.: TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems (2015). tensorflow.​org
2.
Zurück zum Zitat Bar-Shalom, Y.: Tracking and Data Association. Academic Press Professional Inc., San Diego (1987). ISBN 0-120-79760-7MATH Bar-Shalom, Y.: Tracking and Data Association. Academic Press Professional Inc., San Diego (1987). ISBN 0-120-79760-7MATH
4.
Zurück zum Zitat Bishop, C.M.: Mixture density networks. Technical report (1994) Bishop, C.M.: Mixture density networks. Technical report (1994)
7.
8.
Zurück zum Zitat Cox, I.J.: A review of statistical data association techniques for motion correspondence. Int. J. Comput. Vision 10, 53–66 (1993)CrossRef Cox, I.J.: A review of statistical data association techniques for motion correspondence. Int. J. Comput. Vision 10, 53–66 (1993)CrossRef
9.
Zurück zum Zitat Damianou, A., Lawrence, N.: Deep Gaussian processes. In: Artificial Intelligence and Statistics, pp. 207–215, April 2013 Damianou, A., Lawrence, N.: Deep Gaussian processes. In: Artificial Intelligence and Statistics, pp. 207–215, April 2013
10.
Zurück zum Zitat Depeweg, S., Hernández-Lobato, J.M., Doshi-Velez, F., Udluft, S.: Learning and Policy Search in Stochastic Dynamical Systems with Bayesian Neural Networks. arXiv:1605.07127 [cs, stat], May 2016 Depeweg, S., Hernández-Lobato, J.M., Doshi-Velez, F., Udluft, S.: Learning and Policy Search in Stochastic Dynamical Systems with Bayesian Neural Networks. arXiv:​1605.​07127 [cs, stat], May 2016
11.
Zurück zum Zitat Depeweg, S., Hernandez-Lobato, J.-M., Doshi-Velez, F., Udluft, S.: Decomposition of uncertainty in Bayesian deep learning for efficient and risk-sensitive learning. In: International Conference on Machine Learning, pp. 1192–1201 (2018) Depeweg, S., Hernandez-Lobato, J.-M., Doshi-Velez, F., Udluft, S.: Decomposition of uncertainty in Bayesian deep learning for efficient and risk-sensitive learning. In: International Conference on Machine Learning, pp. 1192–1201 (2018)
12.
Zurück zum Zitat Hein, D., Depeweg, S., Tokic, M., et al.: A benchmark environment motivated by industrial control problems. In: 2017 IEEE Symposium Series on Computational Intelligence (SSCI), Honolulu, HI, pp. 1–8. IEEE, November 2017. https://doi.org/10.1109/SSCI.2017.8280935. ISBN 978-1-5386-2726-6 Hein, D., Depeweg, S., Tokic, M., et al.: A benchmark environment motivated by industrial control problems. In: 2017 IEEE Symposium Series on Computational Intelligence (SSCI), Honolulu, HI, pp. 1–8. IEEE, November 2017. https://​doi.​org/​10.​1109/​SSCI.​2017.​8280935. ISBN 978-1-5386-2726-6
13.
Zurück zum Zitat Hensman, J., Fusi, N., Lawrence, N.D.: Gaussian processes for big data. In: Uncertainty in Artificial Intelligence, p. 282. Citeseer (2013) Hensman, J., Fusi, N., Lawrence, N.D.: Gaussian processes for big data. In: Uncertainty in Artificial Intelligence, p. 282. Citeseer (2013)
14.
Zurück zum Zitat Hensman, J., de G. Matthews, A.G., Ghahramani, Z.: Scalable variational Gaussian process classification. J. Mach. Learn. Res. 38, 351–360 (2015) Hensman, J., de G. Matthews, A.G., Ghahramani, Z.: Scalable variational Gaussian process classification. J. Mach. Learn. Res. 38, 351–360 (2015)
15.
Zurück zum Zitat Jacobs, R.A., Jordan, M.I., Nowlan, S.J., Hinton, G.E.: Adaptive mixtures of local experts. Neural Comput. 3(1), 79–87 (1991)CrossRef Jacobs, R.A., Jordan, M.I., Nowlan, S.J., Hinton, G.E.: Adaptive mixtures of local experts. Neural Comput. 3(1), 79–87 (1991)CrossRef
16.
Zurück zum Zitat Kaiser, M., Otte, C., Runkler, T., Ek, C.H.: Bayesian alignments of warped multi-output gaussian processes. In: Bengio, S., Wallach, H., Larochelle, H., et al. (eds.) Advances in Neural Information Processing Systems 31, pp. 6995–7004. Curran Associates Inc., New York (2018) Kaiser, M., Otte, C., Runkler, T., Ek, C.H.: Bayesian alignments of warped multi-output gaussian processes. In: Bengio, S., Wallach, H., Larochelle, H., et al. (eds.) Advances in Neural Information Processing Systems 31, pp. 6995–7004. Curran Associates Inc., New York (2018)
17.
Zurück zum Zitat Kingma, D.P., Salimans, T., Welling, M.: Variational dropout and the local reparameterization trick. In: Cortes, C., Lawrence, N.D., Lee, D.D., Sugiyama, M., Garnett, R. (eds.) Advances in Neural Information Processing Systems 28, pp. 2575–2583. Curran Associates Inc., New York (2015) Kingma, D.P., Salimans, T., Welling, M.: Variational dropout and the local reparameterization trick. In: Cortes, C., Lawrence, N.D., Lee, D.D., Sugiyama, M., Garnett, R. (eds.) Advances in Neural Information Processing Systems 28, pp. 2575–2583. Curran Associates Inc., New York (2015)
18.
Zurück zum Zitat Lázaro-Gredilla, M., Van Vaerenbergh, S., Lawrence, N.D.: Overlapping mixtures of Gaussian processes for the data association problem. Pattern Recogn. 45(4), 1386–1395 (2012)MATHCrossRef Lázaro-Gredilla, M., Van Vaerenbergh, S., Lawrence, N.D.: Overlapping mixtures of Gaussian processes for the data association problem. Pattern Recogn. 45(4), 1386–1395 (2012)MATHCrossRef
19.
Zurück zum Zitat Maddison, C.J., Mnih, A., Teh, Y.W.: The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables. arXiv:1611.00712 [cs, stat], November 2016 Maddison, C.J., Mnih, A., Teh, Y.W.: The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables. arXiv:​1611.​00712 [cs, stat], November 2016
20.
Zurück zum Zitat de G. Matthews, A.G., van der Wilk, M., Nickson, T., et al.: GPflow: a Gaussian process library using TensorFlow. J. Mach. Learn. Res. 18(40), 1–6 (2017)MathSciNetMATH de G. Matthews, A.G., van der Wilk, M., Nickson, T., et al.: GPflow: a Gaussian process library using TensorFlow. J. Mach. Learn. Res. 18(40), 1–6 (2017)MathSciNetMATH
21.
Zurück zum Zitat Rasmussen, C.E., Ghahramani, Z.: Infinite mixtures of Gaussian process experts. In: Dietterich, T.G., Becker, S., Ghahramani, Z. (eds.) Advances in Neural Information Processing Systems 14, pp. 881–888. MIT Press, Cambridge (2002) Rasmussen, C.E., Ghahramani, Z.: Infinite mixtures of Gaussian process experts. In: Dietterich, T.G., Becker, S., Ghahramani, Z. (eds.) Advances in Neural Information Processing Systems 14, pp. 881–888. MIT Press, Cambridge (2002)
22.
Zurück zum Zitat Rezende, D.J., Mohamed, S., Wierstra, D.: Stochastic Backpropagation and Approximate Inference in Deep Generative Models, January 2014 Rezende, D.J., Mohamed, S., Wierstra, D.: Stochastic Backpropagation and Approximate Inference in Deep Generative Models, January 2014
23.
Zurück zum Zitat Salimbeni, H., Deisenroth, M.: Doubly stochastic variational inference for deep Gaussian processes. In: Guyon, I., Luxburg, U.V., Bengio, S., et al. (eds.) Advances in Neural Information Processing Systems 30, pp. 4588–4599. Curran Associates Inc., New York (2017) Salimbeni, H., Deisenroth, M.: Doubly stochastic variational inference for deep Gaussian processes. In: Guyon, I., Luxburg, U.V., Bengio, S., et al. (eds.) Advances in Neural Information Processing Systems 30, pp. 4588–4599. Curran Associates Inc., New York (2017)
24.
Zurück zum Zitat Titsias, M.K.: Variational learning of inducing variables in sparse Gaussian processes. In: AISTATS, vol. 5, pp. 567–574 (2009) Titsias, M.K.: Variational learning of inducing variables in sparse Gaussian processes. In: AISTATS, vol. 5, pp. 567–574 (2009)
25.
Zurück zum Zitat Tresp, V.: Mixtures of Gaussian processes. In: Leen, T.K., Dietterich, T.G., Tresp, V. (eds.) Advances in Neural Information Processing Systems 13, pp. 654–660. MIT Press, Cambridge (2001) Tresp, V.: Mixtures of Gaussian processes. In: Leen, T.K., Dietterich, T.G., Tresp, V. (eds.) Advances in Neural Information Processing Systems 13, pp. 654–660. MIT Press, Cambridge (2001)
Metadaten
Titel
Data Association with Gaussian Processes
verfasst von
Markus Kaiser
Clemens Otte
Thomas A. Runkler
Carl Henrik Ek
Copyright-Jahr
2020
DOI
https://doi.org/10.1007/978-3-030-46147-8_33