Investigation of Factorized Optical Flows as Mid-Level Representations

@article{Yang2022InvestigationOF,
  title={Investigation of Factorized Optical Flows as Mid-Level Representations},
  author={Hsuan-Kung Yang and Tsu-Ching Hsiao and Tingbo Liao and Hsu-Shen Liu and Li-Yuan Tsao and Tzu-Wen Wang and Shan Yang and Yu-Wen Chen and Huang-Ru Liao and Chun-Yi Lee},
  journal={2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
  year={2022},
  pages={746-753}
}
In this paper, we introduce a new concept of incorporating factorized flow maps as mid-level representations, for bridging the perception and the control modules in modular learning based robotic frameworks. To investigate the advantages of factorized flow maps and examine their interplay with the other types of mid-level representations, we further develop a configurable framework, along with four different environments that contain both static and dynamic objects, for analyzing the impacts of… 
View on IEEE
arxiv.org
1 Citations
Background Citations
1

Figures and Tables from this paper

One Citation

Vision based Virtual Guidance for Navigation

The results show that virtual guidance provides the agent with more meaningful navigation information and achieves better performance in terms of path completion rates and navigation efficiency.

References

SHOWING 1-10 OF 33 REFERENCES

Flow-based Intrinsic Curiosity Module

This paper presents a flow-based intrinsic curiosity module (FICM) to exploit the prediction errors from optical flow estimation as exploration bonuses, and proposes the concept of leveraging motion features captured between consecutive observations to evaluate the novelty of observations in an environment.

Representation Flow for Action Recognition

  • A. PiergiovanniM. Ryoo
  • Computer Science
    2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2019
The representation flow layer is a fully-differentiable layer designed to capture the `flow' of any representation channel within a convolutional neural network for action recognition, and newly introduces the concept of learning `flow of flow' representations by stacking multiple representation flow layers.

Driving Policy Transfer via Modularity and Abstraction

This work presents an approach to transferring driving policies from simulation to reality via modularity and abstraction, inspired by classic driving systems and aims to combine the benefits of modular architectures and end-to-end deep learning approaches.

RAFT: Recurrent All-Pairs Field Transforms for Optical Flow

We introduce Recurrent All-Pairs Field Transforms (RAFT), a new deep network architecture for optical flow. RAFT extracts per-pixel features, builds multi-scale 4D correlation volumes for all pairs…

Virtual-to-Real: Learning to Control in Visual Semantic Segmentation

A modular architecture for tackling the virtual-to-real problem, which separates the learning model into a perception module and a control policy module, and uses semantic image segmentation as the meta representation for relating these two modules.

Robust Policies via Mid-Level Visual Representations: An Experimental Study in Manipulation and Navigation

Compared to other approaches for incorporating invariances, such as domain randomization and learning-from-scratch, asynchronously trained mid-level representations scale better: both to harder problems and to larger domain shifts.

Reinforcement Learning with Deep Energy-Based Policies

A method for learning expressive energy-based policies for continuous states and actions, which has been feasible only in tabular domains before, is proposed and a new algorithm, called soft Q-learning, that expresses the optimal policy via a Boltzmann distribution is applied.

Learning to Navigate Using Mid-Level Visual Priors

This large-scale study demonstrates that using mid-level vision results in policies that learn faster, generalize better, and achieve higher final performance, when compared to learning from scratch and/or using state-of-the-art visual and non-visual representation learning methods.

Soft Actor-Critic Algorithms and Applications

Soft Actor-Critic (SAC), the recently introduced off-policy actor-critic algorithm based on the maximum entropy RL framework, achieves state-of-the-art performance, outperforming prior on-policy and off- policy methods in sample-efficiency and asymptotic performance.

Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey

The fundamental background behind sim-to-real transfer in deep reinforcement learning is covered and the main methods being utilized at the moment: domain randomization, domain adaptation, imitation learning, meta-learning and knowledge distillation are overviewed.