Grounded Perceptual Schemas: Developmental Acquisition of Spatial Concepts

Hand-engineered definitions of spatial categories are increasingly seen as brittle and spatial concepts in human interactions may need to learn these in terms of perceptually grounded “image schemas”. Here, we present a developmental approach for the acquisition of grounded spatial schemas in a perceptual agent. We assume a capability for dynamic visual attention, and perceptual notions of wholeness and proximity. We first learn perceptual-object to linguisticname mappings from simple 2D multi-agent visual streams co-occurring with word-separated utterances. Mutual information based statistical measures are seen to be sufficient to identify nominal participants in a simple discourse, based on a synthetic model of dynamic visual attention. Next, we use this knowledge of nominals to ground the semantics of spatial relations in language.We show that a notion of proximity between perceptual objects is sufficient to obtain a pre-verbal notion of graded spatial poses. Once linguistic data is superimposed on this, simple associative structures lead to distinctions such as “in” or “out”. Finally we also show how this can lead to a model of actions, where verbs are learned along with the associated argument structures.