Grounding Action Words in the Sensorimotor Interaction with the World: Experiments with a Simulated iCub Humanoid Robot

  title={Grounding Action Words in the Sensorimotor Interaction with the World: Experiments with a Simulated iCub Humanoid Robot},
  author={Davide Marocco and Angelo Cangelosi and Kerstin Fischer and Tony Belpaeme},
  journal={Frontiers in Neurorobotics},
This paper presents a cognitive robotics model for the study of the embodied representation of action words. The present research will present how an iCub humanoid robot can learn the meaning of action words (i.e. words that represent dynamical events that happen in time) by physically interacting with the environment and linking the effects of its own actions with the behavior observed on the objects before and after the action. The control system of the robot is an artificial neural network… 

Figures from this paper

DAC-h3: A Proactive Robot Cognitive Architecture to Acquire and Express Knowledge About the World and the Self
This paper introduces a cognitive architecture for a humanoid robot to engage in a proactive, mixed-initiative exploration and manipulation of its environment, where the initiative can originate from
Grounding of Language in Sensorimotor World Interaction of a Humanoid Robot , using Neural Networks
Abstract Symbol systems provide techniques to emulate processes of the human mind, such as abstract reasoning, natural language and problem solving, in artificial intelligent agents. To purposefully
Action and Language Integration: From Humans to Cognitive Robots
The papers address the role played not only by sensorimotor and emotional information but also of natural language in conceptual representation, the contextual dependency and high flexibility of the interaction between action, concepts, and language, and the way in which the integration between action and language can be addressed by developmental robotics and Human-Robot Interaction.
C 2011 Logan Niehaus Autonomous Learning of Action-word Semantics in a Humanoid Robot Table of Contents
Presented is a method for online learning of word and action lexicons based on the hidden Markov model that is evaluated through action-word learning experiments implemented on a humanoid robot.
Cognitive Human-Robot Interaction
Research activities in this area include the development of representations and actions that allow robots to participate in joint activities with people; a deeper understanding of human expectations and cognitive responses to robot actions; and, models of joint activity for human–robot interaction.
A deep learning approach for seamless integration of cognitive skills for humanoid robots
The analysis using the dimensionality reduction technique on neuron activation verified that the proposed model was capable of manipulating high dimensional spatio-temporal visuomotor patterns by forming their dynamic link to the actional intention developed in the higher level of the model via iterative learning.
A Probabilistic Approach to Unsupervised Induction of Combinatory Categorial Grammar in Situated Human-Robot Interaction
This paper addresses the challenge of inferring the syntactic structure of language, which embraces grounding parts of speech through visual perception, and induction of Combinatory Categorial Grammar in situated human-robot interaction, which could pave the way towards making a robot able to understand the Syntactic relationships between words.


Learning about objects through action - initial steps towards artificial cognition
It is shown how the humanoid robots can learn how to poke and prod objects to obtain a consistently repeatable effect and to interpret a poking action performed by a human manipulator.
The iCub humanoid robot: an open platform for research in embodied cognition
The iCub is a humanoid robot for research in embodied cognition that will be able to crawl on all fours and sit up to manipulate objects and its hands have been designed to support sophisticate manipulation skills.
On the Role of Robot Simulations in Embodied Cognitive Science
It will be argued in this paper, robot simulations are very useful tools in experimentation with active adaptation of non-trivial environments, an aspect that is still largely ignored in much embodied artificial intelligence research.
An open-source simulator for cognitive robotics research: the prototype of the iCub humanoid robot simulator
The prototype of a new computer simulator for the humanoid robot iCub, developed as part of a joint effort with the European project "ITALK" on the integration and transfer of action and language knowledge in cognitive robots.
Discovering communication
A computational model and a robotic experiment are presented to articulate the hypothesis that children discover communication as a result of exploring and playing with their environment and end up being interested by communication through vocal interactions without having a specific drive for communication.
Evolving grounded communication for robots
Emergence of imitation mediated by objects
It is claimed here that the mapping is not a geometric transformation between bodies but a functional correspondence between someone's action on an object for producing an effect and one's own action on the same or similar object forproducing a similar effect, which is learnable because both tend to utilize the object's affordance in a similar way.
Active perception: a sensorimotor account of object categorization
The analysis of the obtained results shows that evolved individuals always develop a well defined behavioral strategy that allows them to easily and robustly discriminate different objects despite the limitation of their sensory apparatus.
iCub: the design and realization of an open humanoid platform for cognitive and neuroscience research
The design of the mechanisms and structures forming the basic 'body' of the iCub are described and kinematic structures dynamic design criteria, actuator specification and selection, and detailed mechanical and electronic design are considered.
Emergence of Functional Hierarchy in a Multiple Timescale Neural Network Model: A Humanoid Robot Experiment
The proposed network model, coordinating the physical body of a humanoid robot through high-dimensional sensori-motor control, also successfully situated itself within a physical environment and suggests that it is not only the spatial connections between neurons but also the timescales of neural activity that act as important mechanisms leading to functional hierarchy in neural systems.