Hierarchical Reinforcement Learning Framework Towards Multi-Agent Navigation

@article{Ding2018HierarchicalRL,
  title={Hierarchical Reinforcement Learning Framework Towards Multi-Agent Navigation},
  author={Wenhao Ding and Shuaijun Li and Huihuan Qian},
  journal={2018 IEEE International Conference on Robotics and Biomimetics (ROBIO)},
  year={2018},
  pages={237-242}
}
In this paper, we propose a navigation algorithm oriented to multi-agent environment. This algorithm is expressed as a hierarchical framework that contains a Hidden Markov Model (HMM) and a Deep Reinforcement Learning (DRL) structure. For simplification, we term our method Hierarchical Navigation Reinforcement Network (HNRN). In high-level architecture, we train an HMM to evaluate the agents perception to obtain a score. According to this score, adaptive control action will be chosen. While in… 
View on IEEE
arxiv.org
18 Citations
Highly Influential Citations
2
Background Citations
7
Methods Citations
7

Figures and Tables from this paper

18 Citations

Citation Type

Citation Type

Goal-Driven Navigation for Non-holonomic Multi-Robot System by Learning Collision
  • H. Jun, H. Kim, B. H. Lee
  • Computer Science
    2019 International Conference on Robotics and Automation (ICRA)
  • 2019
TLDR
The proposed reinforcement learning based multi-robot collision avoidance approach by learning collision, which learns collision instead of avoiding an obstacle robot, is validated both in the robot simulation and real robot experiment, and compared with the existing collision avoidance method.
Low-Cost Multi-Agent Navigation via Reinforcement Learning With Multi-Fidelity Simulator
TLDR
A Multi-Fidelity Simulator framework to train Multi-Agent Reinforcement Learning (MFS-MARL), reducing the total data cost with samples generated by a low-fidelity simulator and can achieve a 23% cost reduction in multi-agent navigation tasks.
Hierarchical Reinforcement Learning for Autonomous Decision Making and Motion Planning of Intelligent Vehicles
TLDR
A hierarchical reinforcement learning approach for autonomous decision making and motion planning in complex dynamic traffic scenarios by addressing the motion-planning problem in the lateral direction using a dual heuristic programming algorithm learned in a batch-mode manner.
Deep reinforcement learning based mobile robot navigation: A review
TLDR
This paper systematically compares and analyzes the relationship and differences between four typical application scenarios: local obstacle avoidance, indoor navigation, multi-robot navigation, and social navigation; and describes the development of DRL-based navigation.
Learning to Improve Multi-Robot Hallway Navigation
TLDR
This paper develops a methodology to improve existing navigation stacks with regards to both objectives, without tuning their parameters, while preserving their inherent safety control properties, by requiring little computing resources.
Navigation and collision avoidance with human augmented supervisory training and fine tuning via reinforcement learning
TLDR
This study focuses on applying varying levels of human input to an agent to determine how this input affects the agent's performance.
C-Nav: Distributed coordination in crowded multi-agent navigation
Comparative Evaluation for Effectiveness Analysis of Policy Based Deep Reinforcement Learning Approaches
Deep Reinforcement Learning (DRL) has proven to be a very strong technique with results in various applications in recent years. Especially the achievements in the studies in the field of robotics…
Motion planning and control for mobile robot navigation using machine learning: a survey
TLDR
This article surveys recent works that apply machine learning for motion planning and control in mobile robot navigation, within the context of classical navigation systems, to identify common challenges and promising future directions.
Motion Control for Mobile Robot Navigation Using Machine Learning: a Survey
TLDR
This article surveys recent works that apply machine learning for motion control in mobile robot navigation, within the context of classical navigation systems, and identifies common challenges and promising future directions.
...
1
2
...

References

SHOWING 1-10 OF 32 REFERENCES
Motion Planning Among Dynamic, Decision-Making Agents with Deep Reinforcement Learning
TLDR
This work extends the previous approach to develop an algorithm that learns collision avoidance among a variety of types of dynamic agents without assuming they follow any particular behavior rules, and introduces a strategy using LSTM that enables the algorithm to use observations of an arbitrary number of other agents, instead of previous methods that have a fixed observation size.
Towards Optimally Decentralized Multi-Robot Collision Avoidance via Deep Reinforcement Learning
TLDR
This work presents a decentralized sensor-level collision avoidance policy for multi-robot systems, which directly maps raw sensor measurements to an agent's steering commands in terms of movement velocity and demonstrates that the learned policy can be well generalized to new scenarios that do not appear in the entire training period.
Deep-Learned Collision Avoidance Policy for Distributed Multiagent Navigation
TLDR
This work presents a novel end-to-end framework to generate reactive collision avoidance policy for efficient distributed multiagent navigation by formulating an agent's navigation strategy as a deep neural network mapping from the observed noisy sensor measurements to the agent's steering commands in terms of movement velocity.
From perception to decision: A data-driven approach to end-to-end motion planning for autonomous ground robots
TLDR
This work presents the first approach that learns a target-oriented end-to-end navigation model for a robotic platform, and demonstrates that the learned navigation model is directly transferable to previously unseen virtual and, more interestingly, real-world environments.
Hierarchical Reinforcement Learning for Sequencing Behaviors
TLDR
This project explores the capability of deep neural networks to learn and sequence robot skills for navigation, directly using visual input, and aims to jointly learn a set of robot skills and a sequence of these learnt skills to accomplish a given task.
A deep-network solution towards model-less obstacle avoidance
  • L. Tai, Shaohua Li, Ming Liu
  • Computer Science
    2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
  • 2016
TLDR
Inspired by the advantages of deep learning, this work takes indoor obstacle avoidance as example to show the effectiveness of a hierarchical structure that fuses a convolutional neural network (CNN) with a decision process, by which a model-less obstacle avoidance behavior is achieved.
Reciprocal Velocity Obstacles for real-time multi-agent navigation
TLDR
This paper applies the "Reciprocal Velocity Obstacle" concept to navigation of hundreds of agents in densely populated environments containing both static and moving obstacles, and shows that real-time and scalable performance is achieved in such challenging scenarios.
Virtual-to-real deep reinforcement learning: Continuous control of mobile robots for mapless navigation
TLDR
It is shown that, through an asynchronous deep reinforcement learning method, a mapless motion planner can be trained end-to-end without any manually designed features and prior demonstrations.
Combining neural networks and tree search for task and motion planning in challenging environments
TLDR
This work integrates Monte Carlo Tree Search with hierarchical neural net policies trained on expressive LTL specifications to find deep neural networks representing both low-level control policies and task-level “option policies” that achieve high-level goals.
Socially aware motion planning with deep reinforcement learning
TLDR
Using deep reinforcement learning, this work develops a time-efficient navigation policy that respects common social norms and is shown to enable fully autonomous navigation of a robotic vehicle moving at human walking speed in an environment with many pedestrians.
...
1
2
3
4
...