Look Before You Leap: Bridging Model-Free and Model-Based Reinforcement Learning for Planned-Ahead Vision-and-Language Navigation
@article{Wang2018LookBY, title={Look Before You Leap: Bridging Model-Free and Model-Based Reinforcement Learning for Planned-Ahead Vision-and-Language Navigation}, author={Xin Eric Wang and Wenhan Xiong and Hongmin Wang and William Yang Wang}, journal={ArXiv}, year={2018}, volume={abs/1803.07729} }
Existing research studies on vision and language grounding for robot navigation focus on improving model-free deep reinforcement learning (DRL) models in synthetic environments. However, model-free DRL models do not consider the dynamics in the real-world environments, and they often fail to generalize to new scenes. In this paper, we take a radical approach to bridge the gap between synthetic studies and real-world practices---We propose a novel, planned-ahead hybrid reinforcement learning… CONTINUE READING
Figures, Tables, and Topics from this paper
77 Citations
Vision-Language Navigation Policy Learning and Adaptation.
- Computer Science, Medicine
- IEEE transactions on pattern analysis and machine intelligence
- 2020
- 1
- PDF
The Regretful Agent: Heuristic-Aided Navigation Through Progress Estimation
- Computer Science
- 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- 2019
- 49
- PDF
Multi-View Learning for Vision-and-Language Navigation
- Computer Science
- ArXiv
- 2020
- 3
- Highly Influenced
- PDF
Unsupervised Reinforcement Learning of Transferable Meta-Skills for Embodied Navigation
- Computer Science
- 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- 2020
- 4
- PDF
Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation
- Computer Science
- 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- 2019
- 124
- PDF
Learning to Navigate Unseen Environments: Back Translation with Environmental Dropout
- Computer Science
- NAACL-HLT
- 2019
- 57
- Highly Influenced
- PDF
Vision-Based Navigation With Language-Based Assistance via Imitation Learning With Indirect Intervention
- Computer Science, Mathematics
- 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- 2019
- 26
- PDF
Robust Navigation with Language Pretraining and Stochastic Sampling
- Computer Science
- EMNLP/IJCNLP
- 2019
- 15
- PDF
References
SHOWING 1-10 OF 41 REFERENCES
Target-driven visual navigation in indoor scenes using deep reinforcement learning
- Computer Science
- 2017 IEEE International Conference on Robotics and Automation (ICRA)
- 2017
- 730
- PDF
Imagination-Augmented Agents for Deep Reinforcement Learning
- Computer Science, Mathematics
- NIPS
- 2017
- 306
- PDF
Deep visual foresight for planning robot motion
- Computer Science
- 2017 IEEE International Conference on Robotics and Automation (ICRA)
- 2017
- 396
- PDF
Listen, Attend, and Walk: Neural Mapping of Navigational Instructions to Action Sequences
- Computer Science
- AAAI
- 2016
- 146
- PDF
ViZDoom: A Doom-based AI research platform for visual reinforcement learning
- Computer Science, Mathematics
- 2016 IEEE Conference on Computational Intelligence and Games (CIG)
- 2016
- 437
- PDF
Vision-and-Language Navigation: Interpreting Visually-Grounded Navigation Instructions in Real Environments
- Computer Science
- 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
- 2018
- 312
- Highly Influential
- PDF
Mapping Instructions and Visual Observations to Actions with Reinforcement Learning
- Computer Science
- EMNLP
- 2017
- 123
- PDF
Scheduled Policy Optimization for Natural Language Communication with Intelligent Agents
- Computer Science
- IJCAI
- 2018
- 4
- PDF