Grounding words in perception and action: computational insights

@article{Roy2005GroundingWI,
  title={Grounding words in perception and action: computational insights},
  author={Deb K. Roy},
  journal={Trends in Cognitive Sciences},
  year={2005},
  volume={9},
  pages={389-396}
}
  • D. Roy
  • Published 1 August 2005
  • Psychology
  • Trends in Cognitive Sciences
View on Elsevier
media.mit.edu
197 Citations
Highly Influential Citations
6
Background Citations
105
Methods Citations
19

Figures from this paper

197 Citations

Object schemas for grounding language in a responsive robot
An approach is introduced for physically grounded natural language interpretation by robots that reacts appropriately to unanticipated physical changes in the environment and dynamically assimilates…
Grounding human vocabulary in robot perception through interaction
TLDR
Four learning architectures were designed that can be used by robotic agents for long-term and open-ended word and category acquisition and a novel experimental evaluation methodology, that takes into account the openended nature of word learning, is proposed and applied.
Robots That Say “No” Affective Symbol Grounding and the Case of Intent Interpretations
TLDR
This experimental proof-of-concept study shows how nonreferential negation words, typically belonging to a child’s first ten words, may be acquired and corroborates the hypothesis that affect plays a pivotal role in the socially distributed acquisition process where the adept conversation partner provides linguistic interpretations of the affective displays of the less adept speaker.
Grounding Language in Action
TLDR
Findings suggest that what the authors know about action, language and interaction emerge in parallel and have an impact on each other, and this parallel development seems to provide a ground for further mental growth.
The Mechanics of Embodiment: A Dialog on Embodiment and Computational Modeling
TLDR
Six authors from varying backgrounds and approaches address issues concerning the construction of embodied computational models, and illustrate what they view as the critical current and next steps toward mechanistic theories of embodiment.
Grounded Pronoun Learning and Pronoun Reversal
An embodied language-learning system is presented that can learn the correct deictic meanings for the wo rds “I” and “you.” The system uses contextual clues from already understood words and sensory…
Autonomous construction of ecologically and socially relevant semantics
  • M. Takác
  • Sociology
    Cognitive Systems Research
  • 2008
Children in a wonderland: How language and scale errors may be linked
TLDR
It is argued that realising a model of scale errors is essential for understanding the relationship between language, object representation and motor system, and especially their interaction during development.
Chapter 2 – To Cognize is to Categorize: Cognition is Categorization
Embodied object schemas for grounding language use
TLDR
This thesis presents the Object Schema Model (OSM) for grounded language interaction, which is able to interpret basic speech acts that relate to perception of, and actions upon, objects in the robot’s physical environment.
...
...

References

SHOWING 1-10 OF 68 REFERENCES
Grounding language in action
TLDR
The data support an embodied theory of meaning that relates the meaning of sentences to human action, and are inconsistent with theories of language comprehension in which meaning is represented as a set of relations among nodes.
Contentful mental states for robot baby
In this paper we claim that meaningful representations can be learned by programs, although today they are almost always designed by skilled engineers. We discuss several kinds of meaning that…
Mental imagery for a conversational robot
TLDR
A set of representations and procedures are presented that enable a robotic manipulator to maintain a "mental model" of its physical environment by coupling active vision to physical simulation and providing the basis for situated language comprehension and production.
Grounded spoken language acquisition: experiments in word learning
  • D. Roy
  • Computer Science
    IEEE Trans. Multim.
  • 2003
TLDR
Inspired by theories of infant cognition, this work presents a computational model which learns words from untranscribed acoustic and video input which is implemented in a real-time robotic system which performs interactive language learning and understanding.
The Role of Embodied Intention in Early Lexical Acquisition
TLDR
This work is the first model of word learning that not only learns lexical items from raw multisensory signals to closely resemble infant language development from natural environments, but also explores the computational role of social cognitive skills in lexical acquisition.
Mental Models : Towards a Cognitive Science of Language
Mental Models offers nothing less than a unified theory of the major properties of mind: comprehension, inference, and consciousness. In spirited and graceful prose, Johnson-Laird argues that we…
Grounded Semantic Composition for Visual Scenes
TLDR
The implementation of word level visually-grounded semantics and their embedding in a compositional parsing framework is described and how visual context influences the semantics of utterances is revealed.
Emergent constraints on word-learning: a computational perspective
  • T. Regier
  • Linguistics, Psychology
    Trends in Cognitive Sciences
  • 2003
The Brain's concepts: the role of the Sensory-motor system in conceptual knowledge
TLDR
It is proposed that the sensory-motor system has the right kind of structure to characterise both sensory- motor and more abstract concepts, and it is argued against this position using neuroscientific evidence, results from neural computation, and results about the nature of concepts from cognitive linguistics.
An Attentional Constraint on Spatial Meaning
TLDR
This work tested both the perceptual assumption of preferential attention to endpoints, and the linguistic prediction of greater semantic specificity at endpoints.
...
...