Look Before You Leap: Bridging Model-Free and Model-Based Reinforcement Learning for Planned-Ahead Vision-and-Language Navigation

@article{Wang2018LookBY,
  title={Look Before You Leap: Bridging Model-Free and Model-Based Reinforcement Learning for Planned-Ahead Vision-and-Language Navigation},
  author={Xin Eric Wang and Wenhan Xiong and Hongmin Wang and William Yang Wang},
  journal={ArXiv},
  year={2018},
  volume={abs/1803.07729}
}
  • Xin Eric Wang, Wenhan Xiong, +1 author William Yang Wang
  • Published 2018
  • Computer Science
  • ArXiv
  • Existing research studies on vision and language grounding for robot navigation focus on improving model-free deep reinforcement learning (DRL) models in synthetic environments. However, model-free DRL models do not consider the dynamics in the real-world environments, and they often fail to generalize to new scenes. In this paper, we take a radical approach to bridge the gap between synthetic studies and real-world practices---We propose a novel, planned-ahead hybrid reinforcement learning… CONTINUE READING
    77 Citations

    Figures, Tables, and Topics from this paper

    Vision-Language Navigation Policy Learning and Adaptation.
    • 1
    • PDF
    The Regretful Agent: Heuristic-Aided Navigation Through Progress Estimation
    • 49
    • PDF
    Multi-View Learning for Vision-and-Language Navigation
    • 3
    • Highly Influenced
    • PDF
    Sim-to-Real Transfer for Vision-and-Language Navigation
    • 3
    • PDF
    Unsupervised Reinforcement Learning of Transferable Meta-Skills for Embodied Navigation
    • Juncheng Li, X. Wang, +4 authors William Yang Wang
    • Computer Science
    • 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
    • 2020
    • 4
    • PDF
    Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation
    • 124
    • PDF
    Learning to Navigate Unseen Environments: Back Translation with Environmental Dropout
    • 57
    • Highly Influenced
    • PDF
    Vision-Based Navigation With Language-Based Assistance via Imitation Learning With Indirect Intervention
    • 26
    • PDF
    Robust Navigation with Language Pretraining and Stochastic Sampling
    • 15
    • PDF

    References

    SHOWING 1-10 OF 41 REFERENCES
    Target-driven visual navigation in indoor scenes using deep reinforcement learning
    • 730
    • PDF
    Imagination-Augmented Agents for Deep Reinforcement Learning
    • 306
    • PDF
    Value Prediction Network
    • 173
    • PDF
    Deep visual foresight for planning robot motion
    • Chelsea Finn, S. Levine
    • Computer Science
    • 2017 IEEE International Conference on Robotics and Automation (ICRA)
    • 2017
    • 396
    • PDF
    Listen, Attend, and Walk: Neural Mapping of Navigational Instructions to Action Sequences
    • 146
    • PDF
    ViZDoom: A Doom-based AI research platform for visual reinforcement learning
    • 437
    • PDF
    Continuous Deep Q-Learning with Model-based Acceleration
    • 600
    • PDF
    Vision-and-Language Navigation: Interpreting Visually-Grounded Navigation Instructions in Real Environments
    • 312
    • Highly Influential
    • PDF
    Mapping Instructions and Visual Observations to Actions with Reinforcement Learning
    • 123
    • PDF