Top

Published in:

2020 | OriginalPaper | Chapter

Hopfield Networks for Vector Quantization

Authors : C. Bauckhage, R. Ramamurthy, R. Sifa

Published in: Artificial Neural Networks and Machine Learning – ICANN 2020

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

We consider the problem of finding representative prototypes within a set of data and solve it using Hopfield networks. Our key idea is to minimize the mean discrepancy between kernel density estimates of the distributions of data points and prototypes. We show that this objective can be cast as a quadratic unconstrained binary optimization problem which is equivalent to a Hopfield energy minimization problem. This result is of current interest as it suggests that vector quantization can be accomplished via adiabatic quantum computing.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

previous chapter Nonlinear, Nonequilibrium Landscape Approach to Neural Network Dynamics

next chapter Prototype-Based Online Learning on Homogeneously Labeled Streaming Data

Amari, S.I.: Neural theory of association and concept-formation. Biol. Cybern. 26(3), 175–185 (1977). https://doi.org/10.1007/BF00365229CrossRefMATH

Arbel, M., Binkowski, M., Sutherland, D., Gretton, A.: On gradient regularizers for MMD GANs. In: Proceeding NeurIPS (2019)

Bauckhage, C., Brito, E., Cvejoski, K., Ojeda, C., Sifa, R., Wrobel, S.: Ising models for binary clustering via adiabatic quantum computing. In: Pelillo, M., Hancock, E. (eds.) EMMCVPR 2017. LNCS, vol. 10746, pp. 3–17. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-78199-0_1CrossRef

Bauckhage, C., Sanchez, R., Sifa, R.: Problem solving with hopfield networks and adiabatic quantum computing. In: Proceedings IJCNN (2020)

Bauckhage, C., Sifa, R., Dong, T.: Prototypes within minimum enclosing balls. In: Tetko, I.V., Kůrková, V., Karpov, P., Theis, F. (eds.) ICANN 2019. LNCS, vol. 11731, pp. 365–376. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-30493-5_36CrossRef

Bauckhage, C., Thurau, C.: Adapting information theoretic clustering to binary images. In: Proceedings ICPR (2010)

Bian, Z., Chudak, F., Macready, W., Rose, G.: The Ising Model: Teaching an Old Problem New Tricks. Technical report, D-Wave Systems (2010)

Binkowski, M., Sutherland, D., Arbel, M., Gretton, A.: Demystifying MMD GANs. In: Proceedings ICLR (2018)

Caro, M., Aarva, A., Deringer, V., Csanyi, G., Laurila, T.: Reactivity of amorphous carbon surfaces: rationalizing the role of structural motifs in functionalization using machine learning. Chem. Mater. 30(21), 7446–7455 (2018). https://doi.org/10.1021/acs.chemmater.8b03353CrossRef

10.

Cheng, Y.: Mean shift, mode seeking, and clustering. IEEE Trans. Pattern Anal. Mach. Intell. 17(8), 767–776 (1995). https://doi.org/10.1109/34.400568CrossRef

11.

Dasgupta, S., Freund, Y.: Random projection trees for vector quantization. IEEE Trans. Inf. Theory 55(7), 3229–3242 (2009). https://doi.org/10.1109/TIT.2009.2021326MathSciNetCrossRefMATH

12.

Ding, C.: Data clustering: principal components, hopfield and self-aggregation networks. In: Proceedings IJCAI (2003)

13.

Drachen, A., Sifa, R., Thurau, C.: The name in the game: patterns in character names and game tags. Entertain. Comput. 5(1), 21–32 (2014). https://doi.org/10.1016/j.entcom.2014.02.001CrossRef

14.

Dziugaite, G., Roy, D., Ghahramani, Z.: Training generative neural networks via maximum mean discrepancy optimization. In: Proceedings UAI (2015)

15.

Fukunaga, K., Hostelter, L.: The estimation of the gradient of a density function, with applications in pattern recognition. IEEE Trans. Inf. Theory 21(1), 32–40 (1975). https://doi.org/10.1109/TIT.1975.1055330MathSciNetCrossRef

16.

Gersho, A., Gray, R.: Vector Quantization and Signal Compression. Springer, Heidelberg (1991)MATH

17.

Gretton, A., Borgwardt, K., Rasch, M.J., Schölkopf, B., Smola, A.: A kernel two-sample test. J. Mach. Learn. Res. 13(25), 723–773 (2012)MathSciNetMATH

18.

Haykin, S.: Neural Networks and Learning Machines, 3rd edn. Prentice Hall (2008)

19.

Hopfield, J.: Neural networks and physical systems with collective computational abilities. PNAS 79(8), 2554–2558 (1982). https://doi.org/10.1073/pnas.79.8.2554MathSciNetCrossRefMATH

20.

Jegelka, S., Gretton, A., Schölkopf, B., Sriperumbudur, B.K., von Luxburg, U.: Generalized clustering via kernel embeddings. In: Mertsching, B., Hund, M., Aziz, Z. (eds.) KI 2009. LNCS (LNAI), vol. 5803, pp. 144–152. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-04617-9_19CrossRef

21.

Johnson, M., et al.: Quantum annealing with manufactured spins. Nature 473(7346), 194–198 (2011). https://doi.org/10.1038/nature10012CrossRef

22.

Kohonen, T.: Learning vector quantization. In: Self-Organizing Maps. Springer Series in Information Sciences, vol. 30, pp. 175–189. Springer, Heidelberg (1995). https://doi.org/10.1007/978-3-642-97610-0_6

23.

Lehn-Schioler, T., Hedge, A., Erdogmus, D., Principe, J.: Vector quantization using information theoretic concepts. Nat. Comput. 4(1), 39–51 (2005). https://doi.org/10.1007/s11047-004-9619-8MathSciNetCrossRefMATH

24.

Li, C.L., Chang, W.C., Cheng, Y., Tang, Y., Poczos, B.: MMD GAN: towards Deeper understanding of moment matching network. In: Proceedings NIPS (2017)

25.

Li, Y., Swersky, K., Zemel, R.: Generative moment matching networks. In: Proceedings ICML (2015)

26.

Linde, Y., Buzo, A., Gray, R.: An algorithm for vector quantizer design. IEEE Trans. Commun. 28(1), 84–95 (1980). https://doi.org/10.1109/TCOM.1980.1094577CrossRef

27.

Lucas, A.: Ising formulations of many NP problems. Front. Phys. 2, 5:1–5:15 (2014). https://doi.org/10.3389/fphy.2014.00005CrossRef

28.

MacQueen, J.: Some methods for classification and analysis of multivariate observations. In: Proceedings Berkeley Symposium on Mathematical Statistics and Probability (1967)

29.

Martinetz, T., Berkovich, S., Schulten, K.: Neural-gas network for vector quantization and its application to time-series prediction. IEEE Trans. Neural Netw. 4(4), 558–569 (1993). https://doi.org/10.1109/72.238311CrossRef

30.

Molina, A., Vergari, A., Di Mauro, N., Natarajan, S., Esposito, F., Kersting, K.: Mixed sum-product networks: a deep architecture for hybrid domains. In: Proceedings AAAI (2018)

31.

Rao, S., Han, S., Principe, J.: Information theoretic vector quantization with fixed point updates. In: Proceedings IJCNN (2007)

32.

Rojas, R.: Neural Networks – A Systematic Introduction. Springer, Heidelberg (1996)MATH

33.

Sifa, R.: An overview of Frank-Wolfe optimization for stochasticity constrained interpretable matrix and tensor factorization. In: Kůrková, V., Manolopoulos, Y., Hammer, B., Iliadis, L., Maglogiannis, I. (eds.) ICANN 2018. LNCS, vol. 11140, pp. 369–379. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01421-6_36CrossRef

34.

Sifa, R., Bauckhage, C.: Online \(k\)-Maxoids clustering. In: Proceedings DSAA (2017)

35.

Soper, A.: A higher order hopfield network for vector quantisation. Neural Comput. Appl. 7, 99–106 (1998). https://doi.org/10.1007/BF01414161CrossRefMATH

36.

Xu, J.W., Paiva, A., Park, I., Principe, J.: A reproducing kernel Hilbert space framework for information-theoretic learning. IEEE Trans. Signal Process. 56(12), 5891–5902 (2008). https://doi.org/10.1109/TSP.2008.2005085MathSciNetCrossRefMATH

Title: Hopfield Networks for Vector Quantization
Authors: C. Bauckhage
R. Ramamurthy
R. Sifa
Publisher: Springer International Publishing
Book: Artificial Neural Networks and Machine Learning – ICANN 2020
Print ISBN: 978-3-030-61615-1

Electronic ISBN: 978-3-030-61616-8

Copyright Year: 2020
DOI: https://doi.org/10.1007/978-3-030-61616-8_16

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partner