• Corpus ID: 237532669

End-to-End Partially Observable Visual Navigation in a Diverse Environment

  title={End-to-End Partially Observable Visual Navigation in a Diverse Environment},
  author={Bo Ai and Wei Gao and Vinay and David Hsu},
  • Bo Ai, Wei Gao, +1 author David Hsu
  • Published 16 September 2021
  • Computer Science
  • ArXiv
How can a robot navigate successfully in a rich and diverse environment, indoors or outdoors, along an office corridor or a trail in the park, on the flat ground, the staircase, or the elevator, etc.? To this end, this work aims at three challenges: (i) complex visual observations, (ii) partial observability of local sensing, and (iii) multimodal navigation behaviors that depend on both the local environment and the high-level goal. We propose a novel neural network (NN) architecture to… 

Figures and Tables from this paper


Target-driven visual navigation in indoor scenes using deep reinforcement learning
This paper proposes an actor-critic model whose policy is a function of the goal as well as the current state, which allows better generalization and proposes the AI2-THOR framework, which provides an environment with high-quality 3D scenes and a physics engine.
Intention-Net: Integrating Planning and Deep Learning for Goal-Directed Autonomous Navigation
A two-level hierarchical approach, which integrates model-free deep learning and model-based path planning, is introduced, which suggests that the learned motion controller is robust against perceptual uncertainty and by integrating with a path planner, it generalizes effectively to new environments and goals.
End to End Learning for Self-Driving Cars
A convolutional neural network is trained to map raw pixels from a single front-facing camera directly to steering commands and it is argued that this will eventually lead to better performance and smaller systems.
End-to-End Driving Via Conditional Imitation Learning
This work evaluates different architectures for conditional imitation learning in vision-based driving and conducts experiments in realistic three-dimensional simulations of urban driving and on a 1/5 scale robotic truck that is trained to drive in a residential area.
End-to-end Multi-Modal Multi-Task Vehicle Control for Self-Driving Cars with Visual Perceptions
This work proposes a multi-modal multi-task network to predict speed values and steering angles by taking previous feedback speeds and visual recordings as inputs and improves the failure data synthesis methods to solve the problem of error accumulation in real road tests.
Off-Road Obstacle Avoidance through End-to-End Learning
A vision-based obstacle avoidance system for off-road mobile robots that is trained from end to end to map raw input images to steering angles and exhibits an excellent ability to detect obstacles and navigate around them in real time at speeds of 2 m/s.
DeepLanes: End-To-End Lane Position Estimation Using Deep Neural Networks
An approach to estimate lane positions directly using a deep neural network that operates on images from laterally-mounted down-facing cameras, able to estimate the position of a lane marker with sub-centimeter accuracy on an embedded automotive platform, requiring no pre- or post-processing.
Deep Recurrent Q-Learning for Partially Observable MDPs
The effects of adding recurrency to a Deep Q-Network is investigated by replacing the first post-convolutional fully-connected layer with a recurrent LSTM, which successfully integrates information through time and replicates DQN's performance on standard Atari games and partially observed equivalents featuring flickering game screens.
Vision meets robotics: The KITTI dataset
A novel dataset captured from a VW station wagon for use in mobile robotics and autonomous driving research, using a variety of sensor modalities such as high-resolution color and grayscale stereo cameras and a high-precision GPS/IMU inertial navigation system.
ALVINN: An Autonomous Land Vehicle in a Neural Network
ALVINN (Autonomous Land Vehicle In a Neural Network) is a 3-layer back-propagation network designed for the task of road following that can effectively follow real roads under certain field conditions.