Skip to main content
main-content

Tipp

Weitere Kapitel dieses Buchs durch Wischen aufrufen

2017 | OriginalPaper | Buchkapitel

Embedding Learning for Declarative Memories

verfasst von: Volker Tresp, Yunpu Ma, Stephan Baier, Yinchong Yang

Erschienen in: The Semantic Web

Verlag: Springer International Publishing

share
TEILEN

Abstract

The major components of the brain’s declarative or explicit memory are semantic memory and episodic memory. Whereas semantic memory stores general factual knowledge, episodic memory stores events together with their temporal and spatial contexts. We present mathematical models for declarative memories where we consider semantic memory to be represented by triples and episodes to be represented as quadruples i.e., triples in time. E.g., (Jack, receivedDiagnosis, Diabetes, Jan1) states that Jack was diagnosed with diabetes on January 1. Both from a cognitive and a technical perspective, an interesting research question is how declarative data can efficiently be stored and semantically be decoded. We propose that a suitable data representation for episodic event data is a 4-way tensor with dimensions subject, predicate, object, and time. We demonstrate that the 4-way tensor can be decomposed, e.g., using a 4-way Tucker model, which permits semantic decoding of an event, as well as efficient storage. We also propose that semantic memory can be derived from the episodic model by a marginalization of the time dimension, which can be performed efficiently. We argue that the storage of episodic memory typically requires models with a high rank, whereas semantic memory can be modelled with a comparably lower rank. We analyse experimentally the relationship between episodic and semantic memory models and discuss potential relationships to the corresponding brain’s cognitive memories.

Sie möchten Zugang zu diesem Inhalt erhalten? Dann informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 69.000 Bücher
  • über 500 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Testen Sie jetzt 15 Tage kostenlos.

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 50.000 Bücher
  • über 380 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




Testen Sie jetzt 15 Tage kostenlos.

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 58.000 Bücher
  • über 300 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Testen Sie jetzt 15 Tage kostenlos.

Fußnoten
1
This is crucial for investigating human’s semantic memory since the system of memory includes “words and verbal symbols, their meanings and the relations between them” according to Tulving [38]. Large number of relation types contained in the knowledge graph enrich the hidden structure and enhance the complexity of the knowledge graph, and this will give a more realistic and accurate simulation of human’s semantic memory.
 
2
This assignment resembles the active learning process of humans since, during the early stage of learning, we observe the true effects related to a certain subject during a fixed period of time, and store these facts in the form of episodic memory. In our experiment, we highly abstract this learning procedure.
 
Literatur
1.
Zurück zum Zitat Atkinson, R.C., Shiffrin, R.M.: Human memory: a proposed system and its control processes. Psychol. Learn. Motiv. 2, 89–195 (1968). Elsevier CrossRef Atkinson, R.C., Shiffrin, R.M.: Human memory: a proposed system and its control processes. Psychol. Learn. Motiv. 2, 89–195 (1968). Elsevier CrossRef
2.
Zurück zum Zitat Auer, S., Bizer, C., Kobilarov, G., Lehmann, J., Cyganiak, R., Ives, Z.: DBpedia: a nucleus for a web of open data. In: Aberer, K., et al. (eds.) ASWC/ISWC -2007. LNCS, vol. 4825, pp. 722–735. Springer, Heidelberg (2007). doi: 10.​1007/​978-3-540-76298-0_​52 CrossRef Auer, S., Bizer, C., Kobilarov, G., Lehmann, J., Cyganiak, R., Ives, Z.: DBpedia: a nucleus for a web of open data. In: Aberer, K., et al. (eds.) ASWC/ISWC -2007. LNCS, vol. 4825, pp. 722–735. Springer, Heidelberg (2007). doi: 10.​1007/​978-3-540-76298-0_​52 CrossRef
3.
Zurück zum Zitat Baddeley, A.D., Hitch, G., Bower, G.H.: The psychology of learning and motivation (1974) Baddeley, A.D., Hitch, G., Bower, G.H.: The psychology of learning and motivation (1974)
4.
Zurück zum Zitat Bartlett, F.C.: Remembering: A Study in Experimental and Social Psychology, vol. 14. Cambridge University Press, Cambridge (1995) CrossRef Bartlett, F.C.: Remembering: A Study in Experimental and Social Psychology, vol. 14. Cambridge University Press, Cambridge (1995) CrossRef
5.
Zurück zum Zitat Bollacker, K., Evans, C., Paritosh, P., Sturge, T., Taylor, J.: Freebase: a collaboratively created graph database for structuring human knowledge. In: SIGMOD (2008) Bollacker, K., Evans, C., Paritosh, P., Sturge, T., Taylor, J.: Freebase: a collaboratively created graph database for structuring human knowledge. In: SIGMOD (2008)
6.
Zurück zum Zitat Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. In: Advances in Neural Information Processing Systems 26 (2013) Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. In: Advances in Neural Information Processing Systems 26 (2013)
7.
Zurück zum Zitat Carlson, A., Betteridge, J., Kisiel, B., Settles, B., Hruschka Jr., E.R., Mitchell, T.M.: Toward an architecture for never-ending language learning. In: AAAI, vol. 5, p. 3 (2010) Carlson, A., Betteridge, J., Kisiel, B., Settles, B., Hruschka Jr., E.R., Mitchell, T.M.: Toward an architecture for never-ending language learning. In: AAAI, vol. 5, p. 3 (2010)
8.
Zurück zum Zitat Conway, M.A.: Episodic memories. Neuropsychologia 47(11), 2305–2313 (2009). Elsevier CrossRef Conway, M.A.: Episodic memories. Neuropsychologia 47(11), 2305–2313 (2009). Elsevier CrossRef
9.
Zurück zum Zitat Ebbinghaus, H.: Über das Gedächtnis: Untersuchungen zur experimentellen Psychologie. Duncker & Humblot (1885) Ebbinghaus, H.: Über das Gedächtnis: Untersuchungen zur experimentellen Psychologie. Duncker & Humblot (1885)
10.
Zurück zum Zitat Gazzaniga, M.S., Ivry, R.B., Mangun, G.R.: Cognitive Neuroscience: The Biology of the Mind. Norton, New York (2013) Gazzaniga, M.S., Ivry, R.B., Mangun, G.R.: Cognitive Neuroscience: The Biology of the Mind. Norton, New York (2013)
11.
Zurück zum Zitat Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: AISTATS, vol. 9, pp. 249–256 (2010) Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: AISTATS, vol. 9, pp. 249–256 (2010)
12.
Zurück zum Zitat Gluck, M.A., Mercado, E., Myers, C.E.: Learning and Memory: From Brain to Behavior. Palgrave Macmillan, New York (2013) Gluck, M.A., Mercado, E., Myers, C.E.: Learning and Memory: From Brain to Behavior. Palgrave Macmillan, New York (2013)
13.
Zurück zum Zitat Greenberg, D.L., Verfaellie, M.: Interdependence of episodic and semantic memory: evidence from neuropsychology. J. Int. Neuropsychological Soc. 16(05), 748–753 (2010). Cambridge Univ Press CrossRef Greenberg, D.L., Verfaellie, M.: Interdependence of episodic and semantic memory: evidence from neuropsychology. J. Int. Neuropsychological Soc. 16(05), 748–753 (2010). Cambridge Univ Press CrossRef
14.
Zurück zum Zitat Gutierrez, C., Hurtado, C.A., Vaisman, A.: Introducing time into RDF. IEEE Trans. Knowl. Data Eng. 19(2) (2007). IEEE CrossRef Gutierrez, C., Hurtado, C.A., Vaisman, A.: Introducing time into RDF. IEEE Trans. Knowl. Data Eng. 19(2) (2007). IEEE CrossRef
15.
Zurück zum Zitat Hoffart, J., Suchanek, F.M., Berberich, K., Lewis-Kelham, E., De Melo, G., Weikum, G.: Yago2: exploring and querying world knowledge in time, space, context, and many languages. In: WWW. ACM (2011) Hoffart, J., Suchanek, F.M., Berberich, K., Lewis-Kelham, E., De Melo, G., Weikum, G.: Yago2: exploring and querying world knowledge in time, space, context, and many languages. In: WWW. ACM (2011)
16.
Zurück zum Zitat Huth, A.G., de Heer, W.A., Griffiths, T.L., Theunissen, F.E., Gallant, J.L.: Natural speech reveals the semantic maps that tile human cerebral cortex. Nature 532(7600), 453–458 (2016). Nature Publishing Group CrossRef Huth, A.G., de Heer, W.A., Griffiths, T.L., Theunissen, F.E., Gallant, J.L.: Natural speech reveals the semantic maps that tile human cerebral cortex. Nature 532(7600), 453–458 (2016). Nature Publishing Group CrossRef
17.
Zurück zum Zitat Kesner, R.P., Rolls, E.T.: A computational theory of hippocampal function, and tests of the theory: new developments. Prog. Neurobiol. 79(1), 1–48 (2006). Elsevier CrossRef Kesner, R.P., Rolls, E.T.: A computational theory of hippocampal function, and tests of the theory: new developments. Prog. Neurobiol. 79(1), 1–48 (2006). Elsevier CrossRef
19.
Zurück zum Zitat Krompaß, D., Jiang, X., Nickel, M., Tresp, V.: Probabilistic latent-factor database models. In: ECML PKDD (2014) Krompaß, D., Jiang, X., Nickel, M., Tresp, V.: Probabilistic latent-factor database models. In: ECML PKDD (2014)
20.
Zurück zum Zitat Kumar, A., Irsoy, O., Su, J., Bradbury, J., English, R., Pierce, B., Ondruska, P., Gulrajani, I., Socher, R.: Ask me anything: dynamic memory networks for natural language processing. arXiv:​1506.​07285 (2015) Kumar, A., Irsoy, O., Su, J., Bradbury, J., English, R., Pierce, B., Ondruska, P., Gulrajani, I., Socher, R.: Ask me anything: dynamic memory networks for natural language processing. arXiv:​1506.​07285 (2015)
21.
Zurück zum Zitat Loftus, E., Ketcham, K.: The Myth of Repressed Memory: False Memories and Allegations of Sexual Abuse. Macmillan, New York (1996) Loftus, E., Ketcham, K.: The Myth of Repressed Memory: False Memories and Allegations of Sexual Abuse. Macmillan, New York (1996)
22.
Zurück zum Zitat McClelland, J.L., McNaughton, B.L., O’Reilly, R.C.: Why there are complementary learning systems in the hippocampus and neocortex. Psychol. Rev. 102(3), 419 (1995). American Psychological Association CrossRef McClelland, J.L., McNaughton, B.L., O’Reilly, R.C.: Why there are complementary learning systems in the hippocampus and neocortex. Psychol. Rev. 102(3), 419 (1995). American Psychological Association CrossRef
23.
Zurück zum Zitat Morton, N.W.: Interactions between episodic and semantic memory. Technical report, Vanderbilt Computational Memory Lab (2013) Morton, N.W.: Interactions between episodic and semantic memory. Technical report, Vanderbilt Computational Memory Lab (2013)
24.
Zurück zum Zitat Nickel, M., Tresp, V., Kriegel, H.-P.: A three-way model for collective learning on multi-relational data. In: ICML (2011) Nickel, M., Tresp, V., Kriegel, H.-P.: A three-way model for collective learning on multi-relational data. In: ICML (2011)
25.
Zurück zum Zitat Nickel, M., Tresp, V., Kriegel, H.-P.: Factorizing YAGO: scalable machine learning for linked data. In: WWW (2012) Nickel, M., Tresp, V., Kriegel, H.-P.: Factorizing YAGO: scalable machine learning for linked data. In: WWW (2012)
26.
Zurück zum Zitat Nickel, M., Murphy, K., Tresp, V., Gabrilovich, E.: A review of relational machine learning for knowledge graphs: from multi-relational link prediction to automated knowledge graph construction. Proc. IEEE 104(1), 11–33 (2016). IEEE CrossRef Nickel, M., Murphy, K., Tresp, V., Gabrilovich, E.: A review of relational machine learning for knowledge graphs: from multi-relational link prediction to automated knowledge graph construction. Proc. IEEE 104(1), 11–33 (2016). IEEE CrossRef
27.
Zurück zum Zitat Poon, H., Domingos, P.: Sum-product networks: a new deep architecture. In: ICCV (2011) Poon, H., Domingos, P.: Sum-product networks: a new deep architecture. In: ICCV (2011)
28.
Zurück zum Zitat Quiroga, R.Q.: Concept cells: the building blocks of declarative memory functions. Nat. Rev. Neurosci. 13(8), 587–597 (2012). Nature Publishing Group CrossRef Quiroga, R.Q.: Concept cells: the building blocks of declarative memory functions. Nat. Rev. Neurosci. 13(8), 587–597 (2012). Nature Publishing Group CrossRef
29.
Zurück zum Zitat Roediger, H.L., McDermott, K.B.: Creating false memories: remembering words not presented in lists. J. Exp. Psychol. Learn. Mem. Cogn. 21(4), 803 (1995). American Psychological Association CrossRef Roediger, H.L., McDermott, K.B.: Creating false memories: remembering words not presented in lists. J. Exp. Psychol. Learn. Mem. Cogn. 21(4), 803 (1995). American Psychological Association CrossRef
30.
Zurück zum Zitat Rolls, E.T.: A computational theory of episodic memory formation in the hippocampus. Behav. Brain Res. 215(2), 180–196 (2010). Elsevier CrossRef Rolls, E.T.: A computational theory of episodic memory formation in the hippocampus. Behav. Brain Res. 215(2), 180–196 (2010). Elsevier CrossRef
31.
Zurück zum Zitat Rosenblatt, F.: The perceptron: a probabilistic model for information storage and organization in the brain. Psychol. Rev. 65(6), 386 (1958). American Psychological Association CrossRef Rosenblatt, F.: The perceptron: a probabilistic model for information storage and organization in the brain. Psychol. Rev. 65(6), 386 (1958). American Psychological Association CrossRef
33.
Zurück zum Zitat Socher, R., Gershman, S., Sederberg, P., Norman, K., Perotte, A.J., Blei, D.M.: A Bayesian analysis of dynamics in free recall. In: Advances in Neural Information Processing Systems (2009) Socher, R., Gershman, S., Sederberg, P., Norman, K., Perotte, A.J., Blei, D.M.: A Bayesian analysis of dynamics in free recall. In: Advances in Neural Information Processing Systems (2009)
34.
Zurück zum Zitat Squire, L.R.: Memory and Brain. Oxford University Press, New York (1987) Squire, L.R.: Memory and Brain. Oxford University Press, New York (1987)
35.
Zurück zum Zitat Suchanek, F.M., Kasneci, G., Weikum, G.: Yago: a core of semantic knowledge. In: WWW (2007) Suchanek, F.M., Kasneci, G., Weikum, G.: Yago: a core of semantic knowledge. In: WWW (2007)
36.
Zurück zum Zitat Teyler, T.J., DiScenna, P.: The hippocampal memory indexing theory. Behav. Neurosci. 100(2), 147 (1986). American Psychological Association CrossRef Teyler, T.J., DiScenna, P.: The hippocampal memory indexing theory. Behav. Neurosci. 100(2), 147 (1986). American Psychological Association CrossRef
38.
Zurück zum Zitat Tulving, E.: Episodic and semantic memory 1. In: Organization of Memory. Academic, London (1972) Tulving, E.: Episodic and semantic memory 1. In: Organization of Memory. Academic, London (1972)
39.
Zurück zum Zitat Yee, E., Chrysikou, E.G., Thompson-Schill, S.L.: The cognitive neuroscience of semantic memory. In: Oxford Handbook of Cognitive Neuroscience. Oxford University Press (2014) Yee, E., Chrysikou, E.G., Thompson-Schill, S.L.: The cognitive neuroscience of semantic memory. In: Oxford Handbook of Cognitive Neuroscience. Oxford University Press (2014)
Metadaten
Titel
Embedding Learning for Declarative Memories
verfasst von
Volker Tresp
Yunpu Ma
Stephan Baier
Yinchong Yang
Copyright-Jahr
2017
DOI
https://doi.org/10.1007/978-3-319-58068-5_13