• Corpus ID: 195347936

Deep Semantic Matching for Optical Flow

  title={Deep Semantic Matching for Optical Flow},
  author={Min Bai and Wenjie Luo and Kaustav Kundu and Raquel Urtasun},
We tackle the problem of estimating optical flow from a monocular camera in the context of autonomous driving. We build on the observation that the scene is typically composed of a static background, as well as a relatively small number of traffic participants which move rigidly in 3D. We propose to estimate the traffic participants using instance-level segmentation. For each traffic participant, we use the epipolar constraints that govern each independent motion for faster and more accurate… 

Figures and Tables from this paper

Deep Image Homography Estimation

Two convolutional neural network architectures are presented for HomographyNet: a regression network which directly estimates the real-valued homography parameters, and a classification network which produces a distribution over quantized homographies.

Deep corner prediction to rectify tilted license plate images

This work proposes deep neural network models that can locate four corner plate positions, which can then be used to perform the perspective transformation that can been used to rectify plates.

Deep corner prediction to rectify tilted license plate images

This work proposes deep neural network models that can locate four corner plate positions, which can then be used to perform the perspective transformation that can been used to rectify plates.

Building Deep Networks on Grassmann Manifolds

This paper proposes a deep network architecture by generalizing the Euclidean network paradigm to Grassmann manifolds and designs full rank mapping layers to transform input Grassmannian data to more desirable ones, and exploits re-orthonormalization layers to normalize the resulting matrices.

cvpaper.challenge in 2016: Futuristic Computer Vision through 1, 600 Papers Survey

The paper gives futuristic challenges disscussed in the cvpaper.challenge. In 2015 and 2016, we thoroughly study 1,600+ papers in several conferences/journals such as CVPR/ICCV/ECCV/NIPS/PAMI/IJCV.

Deep Discrete Flow

This work presents an efficient way of training a context network with a large receptive field size on top of a local network using dilated convolutions on patches and provides an extensive empirical investigation of network architectures and model parameters.



Efficient Joint Segmentation, Occlusion Labeling, Stereo and Flow Estimation

A new optimization algorithm is proposed for the authors' SLIC-like objective which preserves connecteness of image segments and exploits shape regularization in the form of boundary length and can be achieved an order of magnitude faster than competing approaches.

Robust Monocular Epipolar Flow Estimation

A slanted-plane MRF model is derived which explicitly reasons about the ordering of planes and their physical validity at junctions and a bottom-up grouping algorithm is presented which produces over-segmentations that respect flow boundaries.

Monocular 3D Object Detection for Autonomous Driving

This work proposes an energy minimization approach that places object candidates in 3D using the fact that objects should be on the ground-plane, and achieves the best detection performance on the challenging KITTI benchmark, among published monocular competitors.

A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation

  • N. MayerEddy Ilg T. Brox
  • Computer Science
    2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2016
This paper proposes three synthetic stereo video datasets with sufficient realism, variation, and size to successfully train large networks and presents a convolutional network for real-time disparity estimation that provides state-of-the-art results.

Optical Flow with Semantic Segmentation and Localized Layers

This work exploits recent advances in static semantic scene segmentation to segment the image into objects of different types and poses the flow estimation problem using a novel formulation of localized layers, which addresses limitations of traditional layered models for dealing with complex scene motion.

Object scene flow for autonomous vehicles

A novel model and dataset for 3D scene flow estimation with an application to autonomous driving by representing each element in the scene by its rigid motion parameters and each superpixel by a 3D plane as well as an index to the corresponding object.

Instance-Level Segmentation with Deep Densely Connected MRFs

This paper forms the global labeling problem with a novel densely connected Markov random field and shows how to encode various intuitive potentials in a way that is amenable to efficient mean field inference.

FlowNet: Learning Optical Flow with Convolutional Networks

This paper constructs CNNs which are capable of solving the optical flow estimation problem as a supervised learning task, and proposes and compares two architectures: a generic architecture and another one including a layer that correlates feature vectors at different image locations.

Are we ready for autonomous driving? The KITTI vision benchmark suite

The autonomous driving platform is used to develop novel challenging benchmarks for the tasks of stereo, optical flow, visual odometry/SLAM and 3D object detection, revealing that methods ranking high on established datasets such as Middlebury perform below average when being moved outside the laboratory to the real world.

DeepFlow: Large Displacement Optical Flow with Deep Matching

This work proposes a descriptor matching algorithm, tailored to the optical flow problem, that allows to boost performance on fast motions, and sets a new state-of-the-art on the MPI-Sintel dataset.