Self-organizing maps for storage and transfer of knowledge in reinforcement learning

  title={Self-organizing maps for storage and transfer of knowledge in reinforcement learning},
  author={Thommen George Karimpanal and Roland Bouffanais},
  journal={Adaptive Behavior},
  pages={111 - 126}
The idea of reusing or transferring information from previously learned tasks (source tasks) for the learning of new tasks (target tasks) has the potential to significantly improve the sample efficiency of a reinforcement learning agent. In this work, we describe a novel approach for reusing previously acquired knowledge by using it to guide the exploration of an agent while it learns new tasks. In order to do so, we employ a variant of the growing self-organizing map algorithm, which is… Expand
Model-Based Task Transfer Learning
A model-based task transfer learning (MBTTL) method to find a feasible state-feedback policy for a second task, T1, by using stored data from T2, and results show the effectiveness of the proposed method. Expand
Fully neural object detection solutions for robot soccer
This work presents what is—to their knowledge—the first fully neural vision system for the Nao robot soccer and proposes two novel neural network architectures for semantic segmentation and object detection that ensure low-cost inference, while improving accuracy by exploiting the properties of the environment. Expand
Multi-Agent Reinforcement Learning for Dynamic Ocean Monitoring by a Swarm of Buoys
This paper focuses on robotic swarms that are typically operated and controlled by means of simple swarming behaviors obtained from a subtle, yet ad hoc combination of bio-inspired strategies and proposes a novel and structured approach for area coverage using multi-agent reinforcement learning (MARL). Expand
Multi-agent navigation based on deep reinforcement learning and traditional pathfinding algorithm
A new framework for multi-agent collision avoidance problem that combined traditional pathfinding algorithm and reinforcement learning via a deep neural network trained by reinforcement learning at each time step is developed. Expand
Vision-Based Spacecraft Pose Estimation via a Deep Convolutional Neural Network for Noncooperative Docking Operations
A vision-based pose estimation model that performs image processing via a deep convolutional neural network is constructed that can contribute to spacecraft detection and tracking problems. Expand
The interpretive model of manufacturing: a theoretical framework and research agenda for machine learning in manufacturing
Manufacturing is undergoing a paradigmatic shift as it assimilates and is transformed by machine learning and other cognitive technologies. A new paradigm usually necessitates a new framework toExpand
An end-to-end, datadriven prediction system is proposed to predict the weld penetration status from top-side images during welding to provide more significant improvements during welding using pulsed current where the process becomes highly dynamic. Expand
End-to-end prediction of weld penetration: A deep learning and transfer learning based method
Abstract Weld penetration identification is a long-standing and challenging problem due to the spatial limitation in sensing the back-side of weld joints in practical welding, and the keyExpand
Accurate Recognition of Leukemia Sub-types by Utilizing a Transfer Learned Deep Convolutional Neural Network
Leukemia has been causing more than 350,000 deaths per year despite holding a notable number of studies on the prognosis, diagnosis, and therapy for Leukemia. Computerized Leukemia disclosure mayExpand
Deep Learning to Unveil Correlations between Urban Landscape and Population Health †
A component of such platforms, which couples deep learning analysis of urban geospatial images with healthcare indexes collected by the 500 Cities project, is described, which shows that, in New York City, health care indexes are significantly correlated to the urban landscape. Expand


Self-Organizing Maps as a Storage and Transfer Mechanism in Reinforcement Learning
This work describes an approach to concisely store and represent learned task knowledge, and reuse it by allowing it to guide the exploration of an agent while it learns new tasks using a measure of similarity defined directly in the space of parameterized representations of the value functions. Expand
An automated measure of MDP similarity for transfer in reinforcement learning
A data-driven automated similarity measure for Markov Decision Processes, based on the reconstruction error of a restricted Boltzmann machine that attempts to model the behavioral dynamics of the two MDPs being compared, which can be used to identify similar source tasks for transfer learning. Expand
Horde: a scalable real-time architecture for learning knowledge from unsupervised sensorimotor interaction
Results using Horde on a multi-sensored mobile robot to successfully learn goal-oriented behaviors and long-term predictions from off-policy experience are presented. Expand
Actor-Mimic: Deep Multitask and Transfer Reinforcement Learning
This work defines a novel method of multitask and transfer learning that enables an autonomous agent to learn how to behave in multiple tasks simultaneously, and then generalize its knowledge to new domains, and uses Atari games as a testing environment to demonstrate these methods. Expand
Learning Invariant Feature Spaces to Transfer Skills with Reinforcement Learning
This paper introduces a problem formulation where two agents are tasked with learning multiple skills by sharing information and uses the skills that were learned by both agents to train invariant feature spaces that can be used to transfer other skills from one agent to another. Expand
Successor Features for Transfer in Reinforcement Learning
This work proposes a transfer framework for the scenario where the reward function changes between tasks but the environment's dynamics remain the same, and derives two theorems that set the approach in firm theoretical ground and presents experiments that show that it successfully promotes transfer in practice. Expand
Self-Organizing Neural Networks Integrating Domain Knowledge and Reinforcement Learning
This paper shows how self-organizing neural networks designed for online and incremental adaptation can integrate domain knowledge and RL, and proposes a vigilance adaptation and greedy exploitation strategy to maximize exploitation of the inserted domain knowledge while retaining the plasticity of learning and using new knowledge. Expand
Transfer Reinforcement Learning with Shared Dynamics
This article addresses a particular Transfer Reinforcement Learning problem: when dynamics do not change from one task to another, and only the reward function does, and relies on the optimism in the face of uncertainty principle and to use upper bound reward estimates. Expand
Applications of the self-organising map to reinforcement learning
  • A. Smith
  • Computer Science, Medicine
  • Neural Networks
  • 2002
It is concluded that the SOM is a useful tool for providing real-time, on-line generalisation in RL problems in which the latent dimensionalities of the state and action spaces are small. Expand
The two-dimensional organization of behavior
A fully autonomous multi-modular system designed for the constant accumulation of ever more sophisticated skills (the continual-learning problem) is demonstrated, which can split up a complex task among a large number of simple modules such that nearby modules correspond to similar policies. Expand