Corpus ID: 414469

Functions that Emerge through End-to-end Reinforcement Learning

@article{Shibata2017FunctionsTE,
  title={Functions that Emerge through End-to-end Reinforcement Learning},
  author={Katsunari Shibata},
  journal={ArXiv},
  year={2017},
  volume={abs/1703.02239}
}
  • K. Shibata
  • Published 7 March 2017
  • Computer Science
  • ArXiv
Recently, triggered by the impressive results in TV-games or game of Go by Google DeepMind, end-to-end reinforcement learning (RL) is collecting attentions. Although little is known, the author's group has propounded this framework for around 20 years and already has shown various functions that emerge in a neural network (NN) through RL. In this paper, they are introduced again at this timing. "Function Modularization" approach is deeply penetrated subconsciously. The inputs and outputs for a… Expand
Communications that Emerge through Reinforcement Learning Using a (Recurrent) Neural Network
TLDR
It is shown that a variety of communications emerge through RL using a (recurrent) neural network (NN) and the importance of comprehensive learning and the usefulness of end-to-end RL is shown. Expand
Reinforcement Learning of a Memory Task Using an Echo State Network with Multi-layer Readout
TLDR
It is demonstrated that an RN with MLR can learn a “memory task” through RL with back propagation, and the results suggest that theMLR can make up for the limited computational ability in an RN. Expand
New Reinforcement Learning Using a Chaotic Neural Network for Emergence of "Thinking" - "Exploration" Grows into "Thinking" through Learning -
TLDR
It is shown that a robot with two wheels and two visual sensors reaches a target while avoiding an obstacle after learning though there are still many rooms for improvement. Expand
Q-learning with exploration driven by internal dynamics in chaotic neural network
TLDR
This paper shows chaos-based reinforcement learning (RL) using a chaotic neural network (NN) functions not only with Actor-Critic, but also with Q-learning, and it was confirmed that the agent can adapt to changes in the environment and automatically resume exploration. Expand
Q-learning with exploration driven by internal dynamics in chaos neural network
TLDR
This paper shows chaos-based reinforcement learning (RL) using a chaotic neural network (NN) functions not only with Actor-Critic, but also with Q-learning, and it was confirmed that the agent can adapt to changes in the environment and automatically resume exploration. Expand
Influence of the Chaotic Property on Reinforcement Learning Using a Chaotic Neural Network
TLDR
By reducing the scale of the recurrent connection weights, which is expected to have a deep relation to the chaotic property, the problem was reduced and the learning performance depending on the recurrent weight scale is observed. Expand
Adaptive balancing of exploration and exploitation around the edge of chaos in internal-chaos-based learning
TLDR
It is demonstrated that a chaotic RN can learn without external noise because the output fluctuation originated from its internal chaotic dynamics functions as exploration, and exploration and exploitation are well-balanced around the edge of chaos, which leads to good learning performance. Expand
Sensitivity - Local Index to Control Chaoticity or Gradient Globally
TLDR
Compared with manually fine-tuning the spectral radius of the weight matrix before learning, SAL's continuous nonlinear learning nature prevents loss of sensitivities during learning, resulting in a significant improvement in learning performance. Expand

References

SHOWING 1-10 OF 29 REFERENCES
Human-level control through deep reinforcement learning
TLDR
This work bridges the divide between high-dimensional sensory inputs and actions, resulting in the first artificial agent that is capable of learning to excel at a diverse array of challenging tasks. Expand
Communications that Emerge through Reinforcement Learning Using a (Recurrent) Neural Network
TLDR
It is shown that a variety of communications emerge through RL using a (recurrent) neural network (NN) and the importance of comprehensive learning and the usefulness of end-to-end RL is shown. Expand
Acquisition of deterministic exploration and purposive memory through reinforcement learning with a recurrent neural network
TLDR
In this paper, emergence of deterministic ”exploration” behavior, which is different from the stochastic exploration and needs higher intelligence, is focused on, and it becomes a key point whether the recurrent neural network memorizes necessary information and utilizes it to generate appropriate actions. Expand
Emergence of flexible prediction-based discrete decision making and continuous motion generation through actor-Q-learning
  • K. Shibata, Kenta Goto
  • Computer Science
  • 2013 IEEE Third Joint International Conference on Development and Learning and Epigenetic Robotics (ICDL)
  • 2013
TLDR
To explore the possibility of human-like flexibility in robots, a prediction-required task in which an agent (robot) gets a reward by capturing a moving target that sometimes becomes invisible was learned by reinforcement learning using a recurrent neural network. Expand
Spatial Abstraction and Knowledge Transfer in Reinforcement Learning Using a Multi-Layer Neural Network
Abstraction is a very important function for living things. It generalizes the knowledge obtained through the past experiences and accelerates the learning drastically by applying the generalizedExpand
Acquisition of Flexible Image Recognition by Coupling of Reinforcement Learning and a Neural Network
The authors have proposed a very simple autonomous learning system consisting of one neural network (NN), whose inputs are raw sensor signals and whose outputs are directly passed to actuators asExpand
Acquisition of box pushing by direct-vision-based reinforcement learning
  • K. Shibata, M. Iida
  • Engineering
  • SICE 2003 Annual Conference (IEEE Cat. No.03TH8734)
  • 2003
In this paper, it was confirmed that a real mobile robot with a CCD camera could learn appropriate actions to reach and push a lying box only by direct-vision-based reinforcement learning (RL). InExpand
Direct-Vision-Based Reinforcement Learning in “Going to a Target” Task with an Obstacle and with a Variety of Target Sizes
Two of us has proposed a direct-vision-based reinforcement learning on the neural-network system, in which raw visual sensory signals are directly used as the inputs of the neural network. It hasExpand
Hidden representation after reinforcement learning of hand reaching movement with variable link length
  • K. Shibata, K. Ito
  • Computer Science
  • Proceedings of the International Joint Conference on Neural Networks, 2003.
  • 2003
TLDR
Iriki et al. reported interesting results regarding the visual receptive field of two kinds of neurons in the parietal cortex of a monkey, and it was posited that these neurons contribute to generate the critic output and are obtained through reinforcement learning. Expand
Emergence of Intelligence through Reinforcement Learning with a Neural Network
TLDR
This chapter describes the possibility of the emergence of intelligence or higher functions by the combination of Reinforcement Learning (RL) and a Neural Network (NN), reviewing the author's works up to now. Expand
...
1
2
3
...