Self-organizing maps for storage and transfer of knowledge in reinforcement learning

@article{Karimpanal2019SelforganizingMF,
  title={Self-organizing maps for storage and transfer of knowledge in reinforcement learning},
  author={Thommen George Karimpanal and Roland Bouffanais},
  journal={Adaptive Behavior},
  year={2019},
  volume={27},
  pages={111 - 126}
}
The idea of reusing or transferring information from previously learned tasks (source tasks) for the learning of new tasks (target tasks) has the potential to significantly improve the sample efficiency of a reinforcement learning agent. In this work, we describe a novel approach for reusing previously acquired knowledge by using it to guide the exploration of an agent while it learns new tasks. In order to do so, we employ a variant of the growing self-organizing map algorithm, which is… 

Model-Based Task Transfer Learning

A model-based task transfer learning (MBTTL) method to find a feasible state-feedback policy for a second task, T1, by using stored data from T2, and results show the effectiveness of the proposed method.

Fully neural object detection solutions for robot soccer

This work presents what is—to their knowledge—the first fully neural vision system for the Nao robot soccer and proposes two novel neural network architectures for semantic segmentation and object detection that ensure low-cost inference, while improving accuracy by exploiting the properties of the environment.

Multi-Agent Reinforcement Learning for Dynamic Ocean Monitoring by a Swarm of Buoys

This paper focuses on robotic swarms that are typically operated and controlled by means of simple swarming behaviors obtained from a subtle, yet ad hoc combination of bio-inspired strategies and proposes a novel and structured approach for area coverage using multi-agent reinforcement learning (MARL).

Multi-agent navigation based on deep reinforcement learning and traditional pathfinding algorithm

A new framework for multi-agent collision avoidance problem that combined traditional pathfinding algorithm and reinforcement learning via a deep neural network trained by reinforcement learning at each time step is developed.

Balancing Collective Exploration and Exploitation in Multi-Agent and Multi-Robot Systems: A Review

This review summarizes and categorizes the methods used to control the level of exploration and exploitation carried out by an multi-agent systems, as well as the overall performance of a system with a given cooperative control algorithm.

Balancing Collective Exploration and Exploitation in Multi-Agent and Multi-Robot Systems: A Review

This review summarizes and categorizes the methods used to control the level of exploration and exploitation carried out by an multi-agent systems, as well as the overall performance of a system with a given cooperative control algorithm.

On the Effectiveness of Transfer Learning for Code Search

It is demonstrated that natural language processing models based on the Transformer architecture can be directly applied to source code analysis tasks, such as code search, and the combined use of an information retrieval-based approach followed by a Transformer leads to the best results overall.

References

SHOWING 1-10 OF 40 REFERENCES

Self-Organizing Maps as a Storage and Transfer Mechanism in Reinforcement Learning

This work describes an approach to concisely store and represent learned task knowledge, and reuse it by allowing it to guide the exploration of an agent while it learns new tasks using a measure of similarity defined directly in the space of parameterized representations of the value functions.

An automated measure of MDP similarity for transfer in reinforcement learning

A data-driven automated similarity measure for Markov Decision Processes, based on the reconstruction error of a restricted Boltzmann machine that attempts to model the behavioral dynamics of the two MDPs being compared, which can be used to identify similar source tasks for transfer learning.

Horde: a scalable real-time architecture for learning knowledge from unsupervised sensorimotor interaction

Results using Horde on a multi-sensored mobile robot to successfully learn goal-oriented behaviors and long-term predictions from off-policy experience are presented.

Actor-Mimic: Deep Multitask and Transfer Reinforcement Learning

This work defines a novel method of multitask and transfer learning that enables an autonomous agent to learn how to behave in multiple tasks simultaneously, and then generalize its knowledge to new domains, and uses Atari games as a testing environment to demonstrate these methods.

Learning Invariant Feature Spaces to Transfer Skills with Reinforcement Learning

This paper introduces a problem formulation where two agents are tasked with learning multiple skills by sharing information and uses the skills that were learned by both agents to train invariant feature spaces that can be used to transfer other skills from one agent to another.

Successor Features for Transfer in Reinforcement Learning

This work proposes a transfer framework for the scenario where the reward function changes between tasks but the environment's dynamics remain the same, and derives two theorems that set the approach in firm theoretical ground and presents experiments that show that it successfully promotes transfer in practice.

Self-Organizing Neural Networks Integrating Domain Knowledge and Reinforcement Learning

This paper shows how self-organizing neural networks designed for online and incremental adaptation can integrate domain knowledge and RL, and proposes a vigilance adaptation and greedy exploitation strategy to maximize exploitation of the inserted domain knowledge while retaining the plasticity of learning and using new knowledge.

Transfer Reinforcement Learning with Shared Dynamics

This article addresses a particular Transfer Reinforcement Learning problem: when dynamics do not change from one task to another, and only the reward function does, and relies on the optimism in the face of uncertainty principle and to use upper bound reward estimates.

Applications of the self-organising map to reinforcement learning

The two-dimensional organization of behavior

A fully autonomous multi-modular system designed for the constant accumulation of ever more sophisticated skills (the continual-learning problem) is demonstrated, which can split up a complex task among a large number of simple modules such that nearby modules correspond to similar policies.