OAS-Net: Occlusion Aware Sampling Network for Accurate Optical Flow

@article{Kong2021OASNetOA,
  title={OAS-Net: Occlusion Aware Sampling Network for Accurate Optical Flow},
  author={Lingtong Kong and Xiaohang Yang and Jie Yang},
  journal={ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
  year={2021},
  pages={2475-2479}
}
  • Lingtong Kong, Xiaohang Yang, Jie Yang
  • Published 31 January 2021
  • Computer Science
  • ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
Optical flow estimation is an essential step for many real-world computer vision tasks. Existing deep networks have achieved satisfactory results by mostly employing a pyramidal coarse-to-fine paradigm, where a key process is to adopt warped target feature based on previous flow prediction to correlate with source feature for building 3D matching cost volume. However, the warping operation can lead to troublesome ghosting problem that results in ambiguity. Moreover, occluded areas are treated… 
View on IEEE
View With Semantic Reader

Figures and Tables from this paper

References

SHOWING 1-10 OF 17 REFERENCES
Iterative Residual Refinement for Joint Optical Flow and Occlusion Estimation
  • Junhwa Hur, S. Roth
  • Computer Science
    2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2019
TLDR
This work proposes an iterative residual refinement (IRR) scheme based on weight sharing that can be combined with several backbone networks and achieves state-of-the-art results for both optical flow and occlusion estimation across several standard datasets.
LiteFlowNet: A Lightweight Convolutional Neural Network for Optical Flow Estimation
TLDR
This paper presents an alternative network that attains performance on par with FlowNet2 on the challenging Sintel final pass and KITTI benchmarks, while being 30 times smaller in the model size and 1.36 times faster in the running speed.
Devon: Deformable Volume Network for Learning Optical Flow
TLDR
A new neural network module, Deformable Cost Volume, is proposed which can estimate multi-scale optical flow in a single high resolution and achieves comparable results to the state-of-the-art methods in public benchmarks.
Fdflownet: Fast Optical Flow Estimation Using A Deep Lightweight Network
TLDR
This work presents a lightweight yet effective model for real-time optical flow estimation, termed FDFlowNet (fast deep flownet), which achieves better or similar accuracy on the challenging KITTI and Sintel benchmarks while being about 2 times faster than PWC-Net.
FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks
TLDR
The concept of end-to-end learning of optical flow is advanced and it work really well, and faster variants that allow optical flow computation at up to 140fps with accuracy matching the original FlowNet are presented.
Optical Flow Estimation Using a Spatial Pyramid Network
TLDR
The Spatial Pyramid Network (SPyNet) is much simpler and 96% smaller than FlowNet in terms of model parameters, which makes it more efficient and appropriate for embedded applications.
A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation
  • N. Mayer, Eddy Ilg, T. Brox
  • Computer Science
    2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2016
TLDR
This paper proposes three synthetic stereo video datasets with sufficient realism, variation, and size to successfully train large networks and presents a convolutional network for real-time disparity estimation that provides state-of-the-art results.
PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume
TLDR
PWC-Net has been designed according to simple and well-established principles: pyramidal processing, warping, and the use of a cost volume, and outperforms all published optical flow methods on the MPI Sintel final pass and KITTI 2015 benchmarks.
A Naturalistic Open Source Movie for Optical Flow Evaluation
TLDR
A new optical flow data set derived from the open source 3D animated short film Sintel is introduced, which has important features not present in the popular Middlebury flow evaluation: long sequences, large motions, specular reflections, motion blur, defocus blur, and atmospheric effects.
FlowNet: Learning Optical Flow with Convolutional Networks
TLDR
This paper constructs CNNs which are capable of solving the optical flow estimation problem as a supervised learning task, and proposes and compares two architectures: a generic architecture and another one including a layer that correlates feature vectors at different image locations.
...
...