Top

Hint

Swipe to navigate through the chapters of this book

Published in:

Read chapter Read first chapter

2019 | OriginalPaper | Chapter

Deep Archetypal Analysis

Authors : Sebastian Mathias Keller, Maxim Samarin, Mario Wieser, Volker Roth

Published in: Pattern Recognition

Publisher: Springer International Publishing

Abstract

Deep Archetypal Analysis (DeepAA) generates latent representations of high-dimensional datasets in terms of intuitively understandable basic entities called archetypes. The proposed method extends linear Archetypal Analysis (AA), an unsupervised method to represent multivariate data points as convex combinations of extremal data points. Unlike the original formulation, Deep AA is generative and capable of handling side information. In addition, our model provides the ability for data-driven representation learning which reduces the dependence on expert knowledge. We empirically demonstrate the applicability of our approach by exploring the chemical space of small organic molecules. In doing so, we employ the archetype constraint to learn two different latent archetype representations for the same dataset, with respect to two chemical properties. This type of supervised exploration marks a distinct starting point and let us steer de novo molecular design.

To get access to this content you need the following product:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 69.000 Bücher
über 500 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt 90 Tage mit der neuen Mini-Lizenz testen!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 50.000 Bücher
über 380 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt 90 Tage mit der neuen Mini-Lizenz testen!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 58.000 Bücher
über 300 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt 90 Tage mit der neuen Mini-Lizenz testen!

inform now

previous chapter On the Estimation of the Wasserstein Distance in Generative Models

next chapter Single Level Feature-to-Feature Forecasting with Deformable Convolutions

Available only for authorised users

Note that \(i=1..m\) (and not up to n), which reflects that deep neural networks usually require batch-wise training with batch size m.

Alemi, A.A., Fischer, I., Dillon, J.V., Murphy, K.: Deep variational information bottleneck. CoRR abs/1612.00410 (2016). http://arxiv.org/abs/1612.00410

Atkins, P., de Paula, J.: Atkins’ Physical Chemistry. OUP, Oxford (2010)

Bauckhage, C., Manshaei, K.: Kernel archetypal analysis for clustering web search frequency time series. In: 22nd International Conference on Pattern Recognition, pp. 1544–1549, August 2014. https://doi.org/10.1109/ICPR.2014.274

Bauckhage, C., Kersting, K., Hoppe, F., Thurau, C.: Archetypal analysis as an autoencoder. In: Workshop New Challenges in Neural Computation 2015, pp. 8–16, October 2015. https://www.techfak.uni-bielefeld.de/~fschleif/mlr/mlr_03_2015.pdf

Bauckhage, C., Thurau, C.: Making archetypal analysis practical. In: Denzler, J., Notni, G., Süße, H. (eds.) DAGM 2009. LNCS, vol. 5748, pp. 272–281. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-03798-6_28 CrossRef

Cabeza, L.F., et al.: Lithium in thermal energy storage: a state-of-the-art review. Renew. Sustain. Energy Rev. 42, 1106–1112 (2015) CrossRef

Canhasi, E., Kononenko, I.: Weighted hierarchical archetypal analysis for multi-document summarization. Comput. Speech Lang. 37 (2015). https://doi.org/10.1016/j.csl.2015.11.004

Cutler, A., Breiman, L.: Archetypal analysis. Technometrics 36(4), 338–347 (1994). https://doi.org/10.1080/00401706.1994.10485840. http://digitalassets.lib.berkeley.edu/sdtr/ucb/text/379.pdf MathSciNetCrossRefMATH

van Dijk, D., Burkhardt, D., Amodio, M., Tong, A., Wolf, G., Krishnaswamy, S.: Finding archetypal spaces for data using neural networks. arXiv preprint arXiv:1901.09078 (2019)

10.

Chan, B.H.P., Mitchell, D., Cram, L.: Archetypal analysis of galaxy spectra. Mon. Not. Roy. Astron. Soc. 338 (2003). https://doi.org/10.1046/j.1365-8711.2003.06099.x

11.

Hart, Y., et al.: Inferring biological tasks using pareto analysis of high-dimensional data. Nat. Methods 12(3), 233 (2015) CrossRef

12.

Hou, X., Shen, L., Sun, K., Qiu, G.: Deep feature consistent variational autoencoder. In: IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1133–1141. IEEE (2017)

13.

Huggins, P., Pachter, L., Sturmfels, B.: Toward the human genotope. Bull. Math. Biol. 69(8), 2723–2735 (2007). https://doi.org/10.1007/s11538-007-9244-7 MathSciNetCrossRefMATH

14.

Jang, E., Gu, S., Poole, B.: Categorical reparameterization with Gumbel-Softmax. In: International Conference on Learning Representations (ICLR) (2017)

15.

Kaufmann, D., Keller, S., Roth, V.: Copula archetypal analysis. In: Gall, J., Gehler, P., Leibe, B. (eds.) GCPR 2015. LNCS, vol. 9358, pp. 117–128. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24947-6_10 CrossRef

16.

Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. abs/1412.6980 (2014)

17.

Kingma, D.P., Mohamed, S., Rezende, D.J., Welling, M.: Semi-supervised learning with deep generative models. In: Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, 8–13 December 2014, Montreal, pp. 3581–3589 (2014)

18.

Kingma, D.P., Welling, M.: Auto-encoding variational Bayes. CoRR abs/1312.6114 (2013)

19.

Liu, Z., Luo, P., Wang, X., Tang, X.: Deep learning face attributes in the wild. In: Proceedings of International Conference on Computer Vision (ICCV), December 2015

20.

Mørup, M., Hansen, L.K.: Archetypal analysis for machine learning and data mining. Neurocomputing 80, 54–63 (2012) CrossRef

21.

Parbhoo, S., Wieser, M., Roth, V.: Causal deep information bottleneck. arXiv e-prints arXiv:1807.02326, July 2018

22.

Prabhakaran, S., Raman, S., Vogt, J.E., Roth, V.: Automatic model selection in archetype analysis. In: Pinz, A., Pock, T., Bischof, H., Leberl, F. (eds.) DAGM/OAGM 2012. LNCS, vol. 7476, pp. 458–467. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-32717-9_46 CrossRef

23.

Ramakrishnan, R., Dral, P.O., Rupp, M., von Lilienfeld, O.A.: Quantum chemistry structures and properties of 134 kilo molecules. Sci. Data 1 (2014)

24.

Rezende, D., Mohamed, S.: Variational inference with normalizing flows. In: Bach, F., Blei, D. (eds.) Proceedings of the 32nd International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 37, pp. 1530–1538. PMLR, Lille, 07–09 July 2015

25.

Rezende, D.J., Mohamed, S., Wierstra, D.: Stochastic backpropagation and approximate inference in deep generative models 32(2), 1278–1286 (2014)

26.

Ruddigkeit, L., van Deursen, R., Blum, L.C., Reymond, J.L.: Enumeration of 166 billion organic small molecules in the chemical universe database GDB-17. J. Chem. Inf. Model. 52(11), 2864–2875 (2012). https://doi.org/10.1021/ci300415d. pMID: 23088335 CrossRef

27.

Seth, S., Eugster, M.J.A.: Probabilistic archetypal analysis. Mach. Learn. 102(1), 85–113 (2016). https://doi.org/10.1007/s10994-015-5498-8 MathSciNetCrossRefMATH

28.

Shoval, O., et al.: Evolutionary trade-offs, pareto optimality, and thegeometry of phenotype space. Science 336(6085), 1157–1160 (2012). https://doi.org/10.1126/science.1217405. http://science.sciencemag.org/content/336/6085/1157 CrossRef

29.

Steinbeck, C., Han, Y.Q., Kuhn, S., Horlacher, O., Luttmann, E., Willighagen, E.: The Chemistry Development Kit (CDK): an open-source Java library for chemo- and bioinformatics. J. Chem. Inf. Comput. Sci. 43(2), 493–500 (2003) CrossRef

30.

Tinoco, I.: Physical Chemistry: Principles and Applications in Biological Sciences. No. S. 229-313 in Physical Chemistry: Principles and Applications in Biological Sciences. Prentice Hall, Englewood Cliffs (2002)

31.

Tishby, N., Pereira, F.C., Bialek, W.: The information bottleneck method. arXiv preprint arXiv:physics/0004057 (2000)

32.

Wieczorek, A., Wieser, M., Murezzan, D., Roth, V.: Learning sparse latent representations with the deep copula information bottleneck. In: International Conference on Learning Representations (ICLR) (2018)

33.

Wynen, D., Schmid, C., Mairal, J.: Unsupervised learning of artistic styles with archetypal style analysis. In: Advances in Neural Information Processing Systems, pp. 6584–6593 (2018)

Title: Deep Archetypal Analysis
Authors: Sebastian Mathias Keller
Maxim Samarin
Mario Wieser
Volker Roth
Publisher: Springer International Publishing
Book: Pattern Recognition
Print ISBN: 978-3-030-33675-2

Electronic ISBN: 978-3-030-33676-9

Copyright Year: 2019
DOI: https://doi.org/10.1007/978-3-030-33676-9_12

Springer Professional

Hint

Swipe to navigate through the chapters of this book

Abstract

Please log in to get access to this content

To get access to this content you need the following product:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partner