Top

Cognitive Neurodynamics

Published in:

01-12-2009 | Research Article

A psycholinguistic model of natural language parsing implemented in simulated neurons

Author: Christian R. Huyck

Published in: Cognitive Neurodynamics | Issue 4/2009

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

A natural language parser implemented entirely in simulated neurons is described. It produces a semantic representation based on frames. It parses solely using simulated fatiguing Leaky Integrate and Fire neurons, that are a relatively accurate biological model that is simulated efficiently. The model works on discrete cycles that simulate 10 ms of biological time, so the parser has a simple mapping to psychological parsing time. Comparisons to human parsing studies show that the parser closely approximates this data. The parser makes use of Cell Assemblies and the semantics of lexical items is represented by overlapping hierarchical Cell Assemblies so that semantically related items share neurons. This semantic encoding is used to resolve prepositional phrase attachment ambiguities encountered during parsing. Consequently, the parser provides a neurally-based cognitive model of parsing.

previous article Unifying syntactic theory and sentence processing difficulty through a connectionist minimalist parser

next article The Unification Space implemented as a localist neural net: predictions and error-tolerance in a constraint-based parser

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Abeles M, Bergman H, Margalit E, Vaddia E (1993) Spatiotemporal firing patterns in the frontal cortex of behaving monkeys. J Neurophysiol 70(4):1629–1638PubMed

Amit D (1989) Modelling brain function: the world of attractor neural networks. Cambridge University Press, Cambridge

Anderson J, Lebiere C (1998) The atomic components of thought. Lawrence Erlbaum, Mahwah

Anderson J, Lebiere C (2007) How can the human mind occur in the physical universe. Oxford University Press, OxfordCrossRef

Belavkin R, Huyck C (2008) The emergence of rules in cell assemblies of flif neurons. In: Proceedings of the eighteenth European conference on artificial intelligence

Bevan M, Wilson C (1999) Mechanisms underlying spontaneous oscillation and rhythmic firing in rat subthalamic neurons. Neuroscience 19:7617–7628PubMed

Buonomano D (1999) Distinct functional types of associative long-term potentiation in neocortical and hippocampal pyramidal neurons. Neuroscience 19(16):6748–6754PubMed

Carpenter G, Grossberg S (1988) The art of adaptive pattern recognition by a self-organizing neural network. IEEE Comput 21:77–88

Chacron M, Pakdaman K, Longtin A (2003) Interspike interval correlations, memory, adaptation, and refractoriness in a leaky integrate-and-fire model with threshold fatigue. Neural Comput 15:253–278CrossRefPubMed

Chomsky N (1965) Aspects of the theory of syntax. MIT Press, Cambridge

Dayan P, Abbott L (2005) Theoretical neuroscience: computational and mathematical modeling of neural systems. MIT Press, Cambridge

Fan Y, Huyck C (2008) Implementation of finite state automata using flif neurons. In: IEEE systems, man and cybernetics society, pp 74–78

Filmore C (1968) The case for case. In: Back E, Harms R (eds) Universals in linguistic theory. Holt, Rinehart and Winston Inc., New York

Fodor J, Pylyshyn Z (1988) Connectionism and cognitive architecture: a critical analysis. Cognition 28:3–71CrossRefPubMed

Ford M, Bresnan J, Kaplan R (1982) A competence-based theory of syntactic closure. In: Bresnan J (ed) The mental representation of grammatical relations. MIT Press, Cambridge

Harnad S (1990) The symbol grounding problem. Phys D 42:335–346CrossRef

Hebb DO (1949) The organization of behavior. Wiley, New York

Hempel C, Hartman K, Wang X, Turrigiano G, Nelson S (2000) Multiple forms of short-term plasticity at excitatory in rat medial prefrontal cortex. J Neurophysiol 83:3031–3041PubMed

Henderson J (1994) Connectionist syntactic parsing using temporal variable binding. J Psycholinguist Res 23(5):353–379CrossRef

Hodgkin A, Huxley A (1952) A quantitative description of membrane current and its application to conduction and excitation in nerve. J Physiol 117:500–544PubMed

Huyck C (1999) Modelling cell assemblies. Technical report, Middlesex University

Huyck C (2007) Creating hierarchical categories using cell assemblies. Connect Sci 19(1):1–24CrossRef

Huyck C (2008) CABot1: a videogame agent implemented in fLIF neurons. In: IEEE systems, man and cybernetics society, pp 115–120

Huyck C, Belavkin R (2006) Counting with neurons: rule application with nets of fatiguing leaky integrate and fire neurons. In: Proceedings of the seventh international conference on cognitive modelling, pp 142–147

Huyck C, Fan Y (2007) Parsing with fLIF neurons. In: IEEE systems, man and cybernetics society, pp 35–40

Jackendoff R (1977) X-bar syntax: a study of phrase structure. MIT Press, Cambridge

Jackendoff R (2002) Foundations of language: brain, meaning, grammar, evolution. Oxford University Press, Oxford

Kaplan S, Sontag M, Chown E (1991) Tracing recurrent activity in cognitive elements(TRACE): a model of temporal dynamics in a cell assembly. Connect Sci 3:179–206CrossRef

Kempen G, Vosse T (1991) Incremental syntactic tree formation in human sentence processing: a cognitive architecture based on activation decay and simulated annealing. Connect Sci 1:275–292

Khan M, Lester D, Plana L, Rast A, Painkras J, Furber S (2008) SpiNNaker: mapping neural networks onto a massively-parallel chip multiprocessor. In: Proceeding of 2008 international joint conference on neural networks, pp 2849–2856

Knoblauch A, Markert H, Palm G (2004) An associative model of cortical language and action processing. In: Proceedings of the ninth neural computation and psychology workshop, pp 79–83

Lewis R, Vasishth S (2005) An activation-based model of sentence processing as skilled memory retrieval. Cogn Sci 29(3):375–419

MacDonald M, Pearlmutter N, Seidenberg M (1994) Lexical nature of syntactic ambiguity resolution. Psychol Rev 101(4):676–703CrossRefPubMed

McCulloch W, Pitts W (1943) A logical calculus of ideas immanent in nervous activity. Bull Math Biophys 5:115–133CrossRef

Mikkulainen R (1993) Subsymbolic natural language processing. MIT Press, Cambridge

Miller G (1990) WordNet: an on-line lexical database. Int J Lexicogr 3(4):235−312

Milward O, Cooper R (1994) Incremental interpretation: applications, theory and relationships to dynamic semantics. In: 15th international conference of computational linguistics, pp 748–54

Nadh K, Huyck C (2009) Prepositional phrase attachment ambiguity resolution using semantic hierarchies. In: Proceedings of Artificial Intelligence and Applications (AIA 2009). Innsbruck, Austria

Nakov P, Hearst M (2005) Using the web as an implicit training set: application to structural ambiguity resolution. In: Proceedings of HLT-NAACL

Passmore P, Huyck C (2008) Models of cell assembly decay. In: IEEE systems, man and cybernetics society, pp 251–256

Pollard C, Sag I (1994) Head-driven phrase structure grammar. Center for the Study of Language and Information, Stanford

Pulvermuller F (1999) Words in the brain’s language. Behav Brain Sci 22:253–336CrossRefPubMed

Pulvermuller F (2000) Syntactic circuits: how does the brain create serial order in sentences? Brain Lang 71:194–199CrossRefPubMed

Ratnaparkhi A, Reynar J, Roukos S (1994) A maximum entropy model for prepositional phrase attachment. In: Proceedings of the ARPA human language technology workshop, pp 250–252

Rayner K (1998) Eye movements in reading and information processing: 20 years of research. Psychol Bull 124(3):372–422CrossRefPubMed

Roark B, Johnson M (1999) Efficient probabilistic top-down and left-corner parsing. In: 37th meeting of the ACL, pp 421–428

Rolls E (2008) Memory, attention, and decision-making: a unifying computational neuroscience approach. Oxford University Press, Oxford

Shastri L, Aijanagadde V (1993) From simple associations to systematic reasoning: a connectionist representation of rules, variables, and dynamic bindings using temporal synchrony. Behav Brain Sci 16:417–494CrossRef

Shieber S (1986) An introduction to unification-based approaches to grammar. Center for the Study of Language and Information, Stanford

Smolensky P (1988) On the proper treatment of connectionism. Behav Brain Sci 11(1):1–22

Smolensky P (1990) Tensor product variable binding and the representation of symbolic structures in connectionist systems. Artif Intell 46:159–216CrossRef

Tabor W, Tanenhaus M (1999) Dynamical models of sentence processing. Cogn Sci 23(4):491–515CrossRef

Tal D, Schwartz E (1997) Computing with the leaky integrate-and-fire neuron: logarithmic computation and multiplication. Neural Comput 9(2):305–318CrossRefPubMed

van der Velde F, de Kamps M (2006) Neural blackboard architectures of combinatorial structures in cognition. Behav Brain Sci 29:1–72

Van Gompel R, Pickering R, Traxler M (2001) Reanalysis in sentence processing: evidence against current constraint-based and two-stage models. J Mem Lang 45:225–258CrossRef

von der Malsburg C (1981) The correlation theory of brain function. Technical report, Department of Neurobiology, Max-Planck-Institute for Biophysical Chemistry

von der Malsburg C (1986) Am i thinking assemblies? In: Palm G, Aertsen A (eds) Brain theory. Springer-Verlag, Berlin, pp 161–176

Wennekers T, Palm G (2000) Cell assemblies, associative memory and temporal structure in brain signals. In: Miller R (ed) Time and the brain: conceptual advances in brain research, vol 2. Harwood Academic Publishers, Amsterdam, pp 251–274

Title: A psycholinguistic model of natural language parsing implemented in simulated neurons
Author: Christian R. Huyck
Publication date: 01-12-2009
Publisher: Springer Netherlands
Published in: Cognitive Neurodynamics / Issue 4/2009
Print ISSN: 1871-4080
Electronic ISSN: 1871-4099
DOI: https://doi.org/10.1007/s11571-009-9080-6

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Other articles of this Issue 4/2009

Syntactic sequencing in Hebbian cell assemblies

Grammar resources for modelling dialogue dynamically

Dynamic searching in the brain

Unifying syntactic theory and sentence processing difficulty through a connectionist minimalist parser

A dynamical systems perspective on the relationship between symbolic and non-symbolic computation

Memory reconsolidation for natural language processing