Uncertainty Estimates and Multi-hypotheses Networks for Optical Flow

  title={Uncertainty Estimates and Multi-hypotheses Networks for Optical Flow},
  author={Eddy Ilg and {\"O}zg{\"u}n Çiçek and Silvio Galesso and Aaron Klein and Osama Makansi and Frank Hutter and Thomas Brox},
Optical flow estimation can be formulated as an end-to-end supervised learning problem, which yields estimates with a superior accuracy-runtime tradeoff compared to alternative methodology. In this paper, we make such networks estimate their local uncertainty about the correctness of their prediction, which is vital information when building decisions on top of the estimations. For the first time we compare several strategies and techniques to estimate uncertainty in a large-scale computer… 
Real-Time Uncertainty Estimation in Computer Vision via Uncertainty-Aware Distribution Distillation
This work proposes a simple, easy-to-optimize distillation method for learning the conditional predictive distribution of a pre-trained dropout model for fast, sample-free uncertainty estimation in computer vision tasks, and demonstrates it can significantly reduce the inference time, enabling real-time uncertainty quantification.
Learning the Distribution: A Unified Distillation Paradigm for Fast Uncertainty Estimation in Computer Vision
A unified distillation paradigm for learning the conditional predictive distribution of a pre-trained dropout model for fast uncertainty estimation of both aleatoric and epistemic uncertainty at the same time is proposed.
Fast Uncertainty Estimation for Deep Learning Based Optical Flow
A novel approach to reduce the processing time required to derive the estimation uncertainty map in deep learning-based optical flow determination methods by applying a generative model, which is trained by input images and an uncertainty map derived through a Bayesian approach.
PDC-Net+: Enhanced Probabilistic Dense Correspondence Network
The Enhanced Probabilistic Dense Correspondence Network, PDC-Net+, is proposed, capable of estimating accurate dense correspondences along with a reliable confidence map, and a flexible probabilistic approach that jointly learns the flow prediction and its uncertainty is developed.
Learning Accurate Dense Correspondences and When to Trust Them
This work aims to estimate a dense flow field relating two images, coupled with a robust pixel-wise confidence map indicating the reliability and accuracy of the prediction, and develops a flexible probabilistic approach that jointly learns the flow prediction and its uncertainty.
Separable Flow: Learning Motion Cost Volumes for Optical Flow Estimation
This work proposes a separable cost volume module, a drop-in replacement to correlation cost volumes, that uses non-local aggregation layers to exploit global context cues and prior knowledge, in order to disambiguate motions in these regions.
Evaluating Scalable Bayesian Deep Learning Methods for Robust Computer Vision
This work proposes a comprehensive evaluation framework for scalable epistemic uncertainty estimation methods in deep learning and applies this framework to provide the first properly extensive and conclusive comparison of the two current state-of-the- art scalable methods: ensembling and MC-dropout.
Bayesian Deep Networks for Supervised Single-View Depth Learning
It is demonstrated that adding dropout in the encoder leads to better results than adding it in the decoder, the latest being the usual approach in the literature for similar problems.
Bayesian Deep Neural Networks for Supervised Learning of Single-View Depth
This letter evaluates scalable approaches to uncertainty quantification in single-view supervised depth learning, specifically MC dropout and deep ensembles and explores the use of depth uncertainty for pseudo-RGBD ICP and its potential to estimate accurate two-view relative motion with the real scale.
Probabilistic Pixel-Adaptive Refinement Networks
Encoder-decoder networks have found widespread use in various dense prediction tasks. However, the strong reduction of spatial resolution in the encoder leads to a loss of location information as


ProbFlow: Joint Optical Flow and Uncertainty Estimation
A method that jointly predicts optical flow and its underlying uncertainty and derives a variational inference scheme based on mean field, which incorporates best practices from energy minimization and demonstrates the flexibility of the probabilistic approach by applying it to two different energies and on two benchmarks.
Bootstrap optical flow confidence and uncertainty measure
Learning a Confidence Measure for Optical Flow
A supervised learning-based method to estimate a per-pixel confidence for optical flow vectors and can combine the output of several computed flow fields from different algorithms to select the best performing algorithm per pixel.
FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks
The concept of end-to-end learning of optical flow is advanced and it work really well, and faster variants that allow optical flow computation at up to 140fps with accuracy matching the original FlowNet are presented.
What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
A Bayesian deep learning framework combining input-dependent aleatoric uncertainty together with epistemic uncertainty is presented, which makes the loss more robust to noisy data, also giving new state-of-the-art results on segmentation and depth regression benchmarks.
Learning in an Uncertain World: Representing Ambiguity Through Multiple Hypotheses
This work proposes a frame-work for reformulating existing single-prediction models as multiple hypothesis prediction (MHP) models and an associated meta loss and optimization procedure to train them, and finds that MHP models outperform their single-hypothesis counterparts in all cases and expose valuable insights into the variability of predictions.
A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation
  • N. Mayer, Eddy Ilg, T. Brox
  • Computer Science
    2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2016
This paper proposes three synthetic stereo video datasets with sufficient realism, variation, and size to successfully train large networks and presents a convolutional network for real-time disparity estimation that provides state-of-the-art results.
A Statistical Confidence Measure for Optical Flows
The new confidence measure is computed from merely the results of the optical flow estimator and hence can be applied to any optical flow estimation method, covering the range from local parametric to global variational approaches.
FlowNet: Learning Optical Flow with Convolutional Networks
This paper constructs CNNs which are capable of solving the optical flow estimation problem as a supervised learning task, and proposes and compares two architectures: a generic architecture and another one including a layer that correlates feature vectors at different image locations.
Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles
This work proposes an alternative to Bayesian NNs that is simple to implement, readily parallelizable, requires very little hyperparameter tuning, and yields high quality predictive uncertainty estimates.