Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Published in:
Non-exhaustive Learning Using Gaussian Mixture Generative Adversarial Networks Read chapter Read first chapter

2021 | OriginalPaper | Chapter

The Bures Metric for Generative Adversarial Networks

Authors : Hannes De Meulemeester, Joachim Schreurs, Michaël Fanuel, Bart De Moor, Johan A. K. Suykens

Published in: Machine Learning and Knowledge Discovery in Databases. Research Track

Publisher: Springer International Publishing

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Generative Adversarial Networks (GANs) are performant generative methods yielding high-quality samples. However, under certain circumstances, the training of GANs can lead to mode collapse or mode dropping. To address this problem, we use the last layer of the discriminator as a feature map to study the distribution of the real and the fake data. During training, we propose to match the real batch diversity to the fake batch diversity by using the Bures distance between covariance matrices in this feature space. The computation of the Bures distance can be conveniently done in either feature space or kernel space in terms of the covariance and kernel matrix respectively. We observe that diversity matching reduces mode collapse substantially and has a positive effect on sample quality. On the practical side, a very simple training procedure is proposed and assessed on several data sets.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
previous chapter GraphAnoGAN: Detecting Anomalous Snapshots from Attributed Graphs
next chapter Generative Max-Mahalanobis Classifiers for Image Classification, Generation and More
Appendix
Available only for authorised users
Footnotes
1
For simplicity, we omit the normalization by \(\frac{1}{b-1}\) in front of the covariance matrix.
 
Literature
1.
Arjovsky, M., Chintala, S., Bottou, L.: Wasserstein generative adversarial networks. In: Proceedings of the 34th International Conference on Machine Learning (ICML) (2017)
2.
Bhatia, R., Jain, T., Lim, Y.: On the Bures-Wasserstein distance between positive definite matrices. Expositiones Mathematicae 37(2), 165–191 (2019)MathSciNetCrossRef
3.
Binkowski, M., Sutherland, D.J., Arbel, M., Gretton, A.: Demystifying MMD GANs. In: Proceedings of the International Conference on Learning Representations (ICLR) (2018)
4.
Che, T., Li, Y., Jacob, A.P., Bengio, Y., Li, W.: Mode regularized generative adversarial networks. In: Proceedings of the International Conference on Learning Representations (ICLR) (2017)
5.
6.
Dowson, D., Landau, B.: The Fréchet distance between multivariate normal distributions. J. Multivar. Anal. 12(3), 450–455 (1982)CrossRef
7.
Elfeki, M., Couprie, C., Riviere, M., Elhoseiny, M.: GDPP: learning diverse generations using determinantal point processes. In: Proceedings of the 36th International Conference on Machine Learning (ICML) (2019)
8.
Gelbrich, M.: On a formula for the L2 Wasserstein metric between measures on Euclidean and Hilbert spaces. Mathematische Nachrichten 147(1), 185–203 (1990)MathSciNetCrossRef
9.
Ghosh, A., Kulharia, V., Namboodiri, V.P., Torr, P.H., Dokania, P.K.: Multi-agent diverse generative adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2018)
10.
Goodfellow, I., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, vol. 27 (2014)
11.
Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., Courville, A.C.: Improved training of Wasserstein GANs. In: Advances in Neural Information Processing Systems, vol. 31 (2017)
12.
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., Hochreiter, S.: GANs trained by a two time-scale update rule converge to a local nash equilibrium. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
13.
Hong, Y., Horn, R.A.: The Jordan canonical form of a product of a Hermitian and a positive semidefinite matrix. Linear Algebra Appl. 147, 373–386 (1991)MathSciNetCrossRef
14.
Karras, T., Aila, T., Laine, S., Lehtinen, J.: Progressive growing of GANs for improved quality, stability, and variation. In: Proceedings of the International Conference on Learning Representations (ICLR) (2017)
15.
Kingma, D.P., Ba, J.: Adam: a Method for Stochastic Optimization. In: Proceedings of the International Conference on Learning Representations (ICLR) (2015)
16.
Kingma, D.P., Welling, M.: Auto-encoding variational Bayes. In: Proceedings of the International Conference on Learning Representations (ICLR) (2014)
17.
Lee, K.S., Tran, N.T., Cheung, N.M.: InfoMax-GAN: mutual information maximization for improved adversarial image generation. In: NeurIPS 2019 Workshop on Information Theory and Machine Learning (2019)
18.
Li, C.L., Chang, W.C., Cheng, Y., Yang, Y., Poczos, B.: MMD GAN: towards deeper understanding of moment matching network. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
19.
Lin, Z., Khetan, A., Fanti, G., Oh, S.: PacGAN: the power of two samples in generative adversarial networks. In: Advances in Neural Information Processing Systems, vol. 31 (2018)
20.
Lucic, M., Kurach, K., Michalski, M., Gelly, S., Bousquet, O.: Are GANs created equal? A large-scale study. In: Advances in Neural Information Processing Systems, pp. 700–709 (2018)
21.
Massart, E., Absil, P.A.: Quotient geometry with simple geodesics for the manifold of fixed-rank positive-semidefinite matrices. SIAM J. Matrix Anal. Appl. 41(1), 171–198 (2020)MathSciNetCrossRef
22.
Mescheder, L., Nowozin, S., Geiger, A.: The numerics of GANs. In: Proceedings of the 31st International Conference on Neural Information Processing Systems, p. 1823–1833 (2017)
23.
Metz, L., Poole, B., Pfau, D., Sohl-Dickstein, J.: Unrolled generative adversarial networks. In: Proceedings of the International Conference on Learning Representations (ICLR) (2017)
24.
Miyato, T., Kataoka, T., Koyama, M., Yoshida, Y.: Spectral normalization for generative adversarial networks. In: Proceedings of the International Conference on Learning Representations (ICLR) (2018)
25.
Mroueh, Y., Sercu, T., Goel, V.: McGan: mean and covariance feature matching GAN. In: Proceedings of the 34th International Conference on Machine Learning (ICML) (2017)
26.
Nguyen, T., Le, T., Vu, H., Phung, D.: Dual discriminator generative adversarial nets. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
27.
Oh, J.H., Pouryahya, M., Iyer, A., Apte, A.P., Deasy, J.O., Tannenbaum, A.: A novel kernel Wasserstein distance on Gaussian measures: an application of identifying dental artifacts in head and neck computed tomography. Comput. Biol. Med. 120, 103731 (2020)CrossRef
28.
Peyré, G., Cuturi, M., et al.: Computational optimal transport. Found. Trends Mach. Learn. 11(5–6), 355–607 (2019)CrossRef
29.
Peyre, R.: Comparison between \(W_2\) distance and \(H^{-1}\) norm, and localization of Wasserstein distance. ESAIM: COCV 24(4), 1489–1501 (2018)MATH
30.
Radford, A., Metz, L., Chintala, S.: Unsupervised representation learning with deep convolutional generative adversarial networks. In: Proceedings of the International Conference on Learning Representations (ICLR) (2016)
31.
Ren, Y., Zhu, J., Li, J., Luo, Y.: Conditional Generative Moment-Matching Networks. In: Advances in Neural Information Processing Systems, vol. 29 (2016)
32.
Rezende, D.J., Mohamed, S.: Variational inference with normalizing flows. In: Proceedings of the 32th International Conference on Machine Learning (ICML) (2015)
33.
Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., Chen, X.: Improved techniques for training GANs. In: Advances in Neural Information Processing Systems, vol. 29 (2016)
34.
Song, Y., Ermon, S.: Generative modeling by estimating gradients of the data distribution. In: Advances in Neural Information Processing Systems, pp. 11918–11930 (2019)
35.
Srivastava, A., Valkov, L., Russell, C., Gutmann, M.U., Sutton, C.: VEEGAN: reducing mode collapse in GANs using implicit variational learning. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
36.
Wang, W., Sun, Y., Halgamuge, S.: Improving MMD-GAN training with repulsive loss function. In: Proceedings of the International Conference on Learning Representations (ICLR) (2019)
37.
38.
Xiao, C., Zhong, P., Zheng, C.: BourGAN: generative networks with metric embeddings. In: Advances in Neural Information Processing Systems, vol. 32 (2018)
39.
Zhang, H., Zhang, Z., Odena, A., Lee, H.: Consistency regularization for generative adversarial networks. In: Proceedings of the International Conference on Learning Representations (ICLR) (2020)
Metadata
Title
The Bures Metric for Generative Adversarial Networks
Authors
Hannes De Meulemeester
Joachim Schreurs
Michaël Fanuel
Bart De Moor
Johan A. K. Suykens
Copyright Year
2021
Book
Machine Learning and Knowledge Discovery in Databases. Research Track

Print ISBN: 978-3-030-86519-1

Electronic ISBN: 978-3-030-86520-7

Copyright Year: 2021

DOI
https://doi.org/10.1007/978-3-030-86520-7_4

Premium Partner

    Image Credits