Neuro-Symbolic Reinforcement Learning with First-Order Logic

  title={Neuro-Symbolic Reinforcement Learning with First-Order Logic},
  author={Daiki Kimura and Masaki Ono and Subhajit Chaudhury and Ryosuke Kohita and Akifumi Wachi and Don Joven Agravante and Michiaki Tatsubori and Asim Munawar and Alexander G. Gray},
Deep reinforcement learning (RL) methods often require many trials before convergence, and no direct interpretability of trained policies is provided. In order to achieve fast convergence and interpretability for the policy in RL, we propose a novel RL method for text-based games with a recent neurosymbolic framework called Logical Neural Network, which can learn symbolic and interpretable rules in their differentiable network. The method is first to extract first-order logical facts from text… 
1 Citations

Figures and Tables from this paper

Plugin Framework-Based Neuro-Symbolic Grounded Task Planning for Multi-Agent System
  • Jiyoun Moon
  • Medicine
  • 2021
This study introduces a plugin framework in which multiple robots can be involved in task planning in a broad range of areas by combining symbolic and connectionist approaches.


Reinforcement Learning with External Knowledge by using Logical Neural Networks
This paper proposes an integrated method that enables model-free reinforcement learning from external knowledge sources in an LNNs-based logical constrained framework such as action shielding and guide and demonstrates that the method converges faster compared to a model- free reinforcement learning method that doesn’t have such logical constraints.
Asynchronous Methods for Deep Reinforcement Learning
A conceptually simple and lightweight framework for deep reinforcement learning that uses asynchronous gradient descent for optimization of deep neural network controllers and shows that asynchronous actor-critic succeeds on a wide variety of continuous motor control problems as well as on a new task of navigating random 3D mazes using a visual input.
Human-level control through deep reinforcement learning
This work bridges the divide between high-dimensional sensory inputs and actions, resulting in the first artificial agent that is capable of learning to excel at a diverse array of challenging tasks.
Playing Atari with Deep Reinforcement Learning
This work presents the first deep learning model to successfully learn control policies directly from high-dimensional sensory input using reinforcement learning, which outperforms all previous approaches on six of the games and surpasses a human expert on three of them.
DAQN: Deep Auto-encoder and Q-Network
This paper proposes an extended deep reinforcement learning method that is applied a generative model to initialize the network for reducing the number of training trials and trained efficiently on all tasks.
Neural Logic Machines
The Neural Logic Machine is proposed, a neural-symbolic architecture for both inductive learning and logic reasoning that achieves perfect generalization in a number of tasks, from relational reasoning tasks on the family tree and general graphs, to decision making tasks including sorting arrays, finding shortest paths, and playing the blocks world.
Internal Model from Observations for Reward Shaping
This work proposes a novel reinforcement learning method in which the agent learns an internal model of observation on the basis of expert-demonstrated state trajectories to estimate rewards without completely learning the dynamics of the external environment from state-action pairs.
LeDeepChef: Deep Reinforcement Learning Agent for Families of Text-Based Games
This work presents a deep RL agent--LeDeepChef--that shows generalization capabilities to never-before-seen games of the same family with different environments and task descriptions, and uses an actor-critic framework and prune the action-space to build an agent that achieves high scores across a whole family of games.
Simple statistical gradient-following algorithms for connectionist reinforcement learning
This article presents a general class of associative reinforcement learning algorithms for connectionist networks containing stochastic units that are shown to make weight adjustments in a direction that lies along the gradient of expected reinforcement in both immediate-reinforcement tasks and certain limited forms of delayed-reInforcement tasks, and they do this without explicitly computing gradient estimates.
Bootstrapped Q-learning with Context Relevant Observation Pruning to Generalize in Text-based Games
A bootstrapped agent shows improved generalization in solving unseen TextWorld games, using 10x-20x fewer training games compared to previous state-of-the-art methods despite requiring less number of training episodes.