• Corpus ID: 207847608

Language Grounding through Social Interactions and Curiosity-Driven Multi-Goal Learning

@article{Lair2019LanguageGT,
  title={Language Grounding through Social Interactions and Curiosity-Driven Multi-Goal Learning},
  author={Nicolas Lair and C{\'e}dric Colas and R{\'e}my Portelas and Jean-Michel Dussoux and Peter Ford Dominey and Pierre-Yves Oudeyer},
  journal={ArXiv},
  year={2019},
  volume={abs/1911.03219}
}
Autonomous reinforcement learning agents, like children, do not have access to predefined goals and reward functions. They must discover potential goals, learn their own reward functions and engage in their own learning trajectory. Children, however, benefit from exposure to language, helping to organize and mediate their thought. We propose LE2 (Language Enhanced Exploration), a learning algorithm leveraging intrinsic motivations and natural language (NL) interactions with a descriptive social… 

Figures from this paper

Learning with AMIGo: Adversarially Motivated Intrinsic Goals
TLDR
AMIGo is proposed, a novel agent incorporating a goal-generating teacher that proposes Adversarially Motivated Intrinsic Goals to train aGoal-conditioned "student" policy in the absence of (or alongside) environment reward to solve challenging procedurally-generated tasks.
Learning Multi-Objective Curricula for Deep Reinforcement Learning
TLDR
A unified automatic curriculum learning framework to create multi-objective but coherent curricula that are generated by a set of parametric curriculum modules is proposed and a flexible memory mechanism to learn an abstract curriculum is designed, which may otherwise be difficult to design manually.
Touch-based Curiosity for Sparse-Reward Tasks
TLDR
This work uses surprise from mismatches in touch feedback to guide exploration in hard sparse-reward reinforcement learning tasks and compares its cross-modal approach to single-modality (touchor vision-only) approaches as well as other curiosity-based methods and finds that it performs better and is more sample-efficient.
Haptics-based Curiosity for Sparse-reward Tasks
TLDR
This work uses surprise from mismatches in haptics feedback to guide exploration in hard sparse-reward reinforcement learning tasks, and compares its cross-modal approach to single-modality (hapticsor vision-only) approaches as well as other curiosity-based methods and finds that the method performs better and is more sample-efficient.
Automatic Curriculum Learning For Deep RL: A Short Survey
TLDR
The ambition of this work is to present a compact and accessible introduction to the Automatic Curriculum Learning literature and to draw a bigger picture of the current state of the art in ACL to encourage the cross-breeding of existing concepts and the emergence of new ideas.

References

SHOWING 1-10 OF 31 REFERENCES
CURIOUS: Intrinsically Motivated Modular Multi-Goal Reinforcement Learning
TLDR
CURIOUS is proposed, an algorithm that leverages a modular Universal Value Function Approximator with hindsight learning to achieve a diversity of goals of different kinds within a unique policy and an automated curriculum learning mechanism that biases the attention of the agent towards goals maximizing the absolute learning progress.
Intrinsically Motivated Goal Exploration Processes with Automatic Curriculum Learning
TLDR
It is illustrated the computational efficiency of IMGEPs as these robotic experiments use a simple memory-based low-level policy representations and search algorithm, enabling the whole system to learn online and incrementally on a Raspberry Pi 3.
From Language to Goals: Inverse Reinforcement Learning for Vision-Based Instruction Following
TLDR
This work proposes language-conditioned reward learning (LC-RL), which grounds language commands as a reward function represented by a deep neural network, and demonstrates that the model learns rewards that transfer to novel tasks and environments on realistic, high-dimensional visual environments with natural language commands.
Using Natural Language for Reward Shaping in Reinforcement Learning
TLDR
This work proposes the LanguagE-Action Reward Network (LEARN), a framework that maps free-form natural language instructions to intermediate rewards based on actions taken by the agent that can seamlessly be integrated into any standard reinforcement learning algorithm.
CURIOUS: Intrinsically Motivated Multi-Task, Multi-Goal Reinforcement Learning
TLDR
CURIOUS is proposed, an extension of Universal Value Function Approximators that enables intrinsically motivated agents to learn to achieve both multiple tasks and multiple goals within a unique policy, leveraging hindsight learning.
Learning to Understand Goal Specifications by Modelling Reward
TLDR
A framework within which instruction-conditional RL agents are trained using rewards obtained not from the environment, but from reward models which are jointly trained from expert examples, which allows an agent to adapt to changes in the environment without requiring new expert examples.
A Survey of Reinforcement Learning Informed by Natural Language
TLDR
The time is right to investigate a tight integration of natural language understanding into Reinforcement Learning in particular, and the state of the field is surveyed, including work on instruction following, text games, and learning from textual domain knowledge.
Language as an Abstraction for Hierarchical Deep Reinforcement Learning
TLDR
This paper introduces an open-source object interaction environment built using the MuJoCo physics engine and the CLEVR engine and finds that, using the approach, agents can learn to solve to diverse, temporally-extended tasks such as object sorting and multi-object rearrangement, including from raw pixel observations.
Hindsight Experience Replay
TLDR
A novel technique is presented which allows sample-efficient learning from rewards which are sparse and binary and therefore avoid the need for complicated reward engineering and may be seen as a form of implicit curriculum.
Modular active curiosity-driven discovery of tool use
TLDR
The limits of existing goal babbling architectures are shown for efficient exploration in high-dimensional structured sensorimotor spaces, and a novel exploration architecture called Model Babbling is introduced, which exploits efficiently a modular representation of the space of parameterized problems/effects.
...
...