• Corpus ID: 53085944

CURIOUS: Intrinsically Motivated Multi-Task, Multi-Goal Reinforcement Learning

@article{Colas2018CURIOUSIM,
  title={CURIOUS: Intrinsically Motivated Multi-Task, Multi-Goal Reinforcement Learning},
  author={C{\'e}dric Colas and Pierre Fournier and Olivier Sigaud and Pierre-Yves Oudeyer},
  journal={ArXiv},
  year={2018},
  volume={abs/1810.06284}
}
In open-ended and changing environments, agents face a wide range of potential tasks that may or may not come with associated reward functions. [] Key Method This mechanism provides robustness to catastrophic forgetting (by refocusing on tasks where performance decreases) and distracting tasks (by avoiding tasks with no absolute learning progress). Furthermore, we show that having two levels of parameterization (tasks and goals within tasks) enables more efficient learning of skills in an environment with a…

Figures and Tables from this paper

Intrinsically Motivated Goal Exploration Processes with Automatic Curriculum Learning

It is illustrated the computational efficiency of IMGEPs as these robotic experiments use a simple memory-based low-level policy representations and search algorithm, enabling the whole system to learn online and incrementally on a Raspberry Pi 3.

Learning Embodied Agents with Scalably-Supervised Reinforcement Learning

This thesis considers alternative modalities of supervision that can be more scalable and easier to provide from the human user and shows that such supervision can drastically improve the agent’s learning efficiency, enabling the agent to do directed exploration and learning within a large search space of states.

Exploration via Hindsight Goal Generation

HGG is introduced, a novel algorithmic framework that generates valuable hindsight goals which are easy for an agent to achieve in the short term and are also potential for guiding the agent to reach the actual goal in the long term.

Stein Variational Goal Generation For Reinforcement Learning in Hard Exploration Problems

This work proposes a novel approach: Stein Variational Goal Generation (SVGG), which seeks at preferably sampling new goals in the zone of proximal development of the agent, by leveraging a learned model of its abilities, and a goal distribution modeled as particles in the exploration space.

Skew-Fit: State-Covering Self-Supervised Reinforcement Learning

This paper proposes a formal exploration objective for goal-reaching policies that maximizes state coverage and presents an algorithm called Skew-Fit, which enables a real-world robot to learn to open a door, entirely from scratch, from pixels, and without any manually-designed reward function.

Intrinsically Motivated Exploration of Learned Goal Spaces

This article shows that the goal space can be learned using deep representation learning algorithms, effectively reducing the burden of designing goal spaces and paving the way to autonomous learning agents that are able to autonomously build a representation of the world and use this representation to explore the world efficiently.

Curiosity-Driven Multi-Criteria Hindsight Experience Replay

This work is the first to stack more than two blocks using only sparse reward without human demonstrations, and presents a method that combines hindsight with curiosity-driven exploration and curriculum learning in order to solve the challenging sparse-reward block stacking task.

SKEW-FIT: STATE-COVERING SELF-SUPERVISED RE-

This work proposes a formal exploration objective for goal-reaching policies that maximizes state coverage, and presents an algorithm called Skew-Fit, which can learn a variety of manipulation tasks from images, including opening a door with a real robot, entirely from scratch and without any manually-designed reward function.

Autonomous Goal Exploration using Learned Goal Spaces for Visuomotor Skill Acquisition in Robots

Recent results showing the applicability of Intrinsically Motivated Goal Exploration Processes principles on a real-world robotic setup, where a 6-joint robotic arm learns to manipulate a ball inside an arena, by choosing goals in a space learned from its past experience are presented.

Language Grounding through Social Interactions and Curiosity-Driven Multi-Goal Learning

LE2 (Language Enhanced Exploration) is proposed, a learning algorithm leveraging intrinsic motivations and natural language (NL) interactions with a descriptive social partner (SP) that can learn an NL-conditioned reward function to formulate goals for intrinsically motivated goal exploration and learn a goal- Conditioned policy.

References

SHOWING 1-10 OF 34 REFERENCES

Intrinsically Motivated Goal Exploration Processes with Automatic Curriculum Learning

It is illustrated the computational efficiency of IMGEPs as these robotic experiments use a simple memory-based low-level policy representations and search algorithm, enabling the whole system to learn online and incrementally on a Raspberry Pi 3.

Multi-task Deep Reinforcement Learning with PopArt

This work proposes to automatically adapt the contribution of each task to the agent’s updates, so that all tasks have a similar impact on the learning dynamics, and learns a single trained policy that exceeds median human performance on this multi-task domain.

Overcoming Exploration in Reinforcement Learning with Demonstrations

This work uses demonstrations to overcome the exploration problem and successfully learn to perform long-horizon, multi-step robotics tasks with continuous control such as stacking blocks with a robot arm.

Hindsight Experience Replay

A novel technique is presented which allows sample-efficient learning from rewards which are sparse and binary and therefore avoid the need for complicated reward engineering and may be seen as a form of implicit curriculum.

Active learning of inverse models with intrinsically motivated goal exploration in robots

Distral: Robust multitask reinforcement learning

This work proposes a new approach for joint training of multiple tasks, which it refers to as Distral (Distill & transfer learning), and shows that the proposed learning process is more robust and more stable---attributes that are critical in deep reinforcement learning.

Unicorn: Continual Learning with a Universal, Off-policy Agent

A novel agent architecture is proposed called Unicorn, which demonstrates strong continual learning and outperforms several baseline agents on the proposed domain with an implicit sequence of tasks and sparse rewards.

Curiosity Driven Exploration of Learned Disentangled Goal Spaces

It is shown that using a disentangled goal space leads to better exploration performances than an entangled goal space and that the measure of learning progress, used to drive curiosity-driven exploration, can be used simultaneously to discover abstract independently controllable features of the environment.

IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures

A new distributed agent IMPALA (Importance Weighted Actor-Learner Architecture) is developed that not only uses resources more efficiently in single-machine training but also scales to thousands of machines without sacrificing data efficiency or resource utilisation.

Automatic Goal Generation for Reinforcement Learning Agents

This work uses a generator network to propose tasks for the agent to try to achieve, specified as goal states, and shows that, by using this framework, an agent can efficiently and automatically learn to perform a wide set of tasks without requiring any prior knowledge of its environment.