Video Deblurring via Semantic Segmentation and Pixel-Wise Non-linear Kernel

@article{Ren2017VideoDV,
  title={Video Deblurring via Semantic Segmentation and Pixel-Wise Non-linear Kernel},
  author={Wenqi Ren and Jin-shan Pan and Xiaochun Cao and Ming-Hsuan Yang},
  journal={2017 IEEE International Conference on Computer Vision (ICCV)},
  year={2017},
  pages={1086-1094}
}
Video deblurring is a challenging problem as the blur is complex and usually caused by the combination of camera shakes, object motions, and depth variations. Optical flow can be used for kernel estimation since it predicts motion trajectories. However, the estimates are often inaccurate in complex scenes at object boundaries, which are crucial in kernel estimation. In this paper, we exploit semantic segmentation in each blurry frame to understand the scene contents and use different motion… 
Spatio-Temporal Filter Adaptive Network for Video Deblurring
TLDR
The proposed Spatio-Temporal Filter Adaptive Network (STFAN) takes both blurry and restored images of the previous frame as well as blurry image of the current frame as input, and dynamically generates the spatially adaptive filters for the alignment and deblurring.
Joint Stereo Video Deblurring, Scene Flow Estimation and Moving Object Segmentation
TLDR
Under the model, these three tasks are naturally connected and expressed as the parameter estimation of 3D scene structure and camera motion (structure and motion for the dynamic scenes), and the method reaches an energy minimization formulation for joint deblurring, scene flow and segmentation.
Self-supervised Exposure Trajectory Recovery for Dynamic Blur Estimation
TLDR
Under mild constraints, the learned motion offsets can recover dense, (non-)linear exposure trajectories, which significantly reduce temporal disorder and ill-posed problems and further contribute to motion-aware image deblurring and warping-based video extraction from a single blurry image.
Recurrent Video Deblurring with Blur-Invariant Motion Estimation and Pixel Volumes
TLDR
This work presents blur-invariant motion estimation learning to improve motion estimation accuracy between blurry frames, and proposes an effective recurrent video deblurring network that fully exploits deblurred previous frames.
Exposure Trajectory Recovery from Motion Blur.
TLDR
Exposure trajectories are defined, which represent the motion information contained in a blurry image and explain the causes of motion blur, and a novel motion offset estimation framework is proposed to model pixel-wise displacements of the latent sharp image at multiple timepoints.
VDFlow: Joint Learning for Optical Flow and Video Deblurring
TLDR
This paper proposes a united end-to-end network, called VDFlow, for both optical flow estimation and video deblurring simultaneously, and demonstrates that the proposed method performs favorably against the state-of-the-art videodeblurring approaches on challenging blurry videos and improves the performance of Optical flow estimation as well.
Prior-enlightened and Motion-robust Video Deblurring
TLDR
A PRiOr-enlightened and MOTION-robust video deblurring model suitable for challenging blurs, which uses 3D group convolution to efficiently encode heterogeneous prior information, explicitly enhancing the scenes' perception while mitigating the output's artifacts.
Learning Event-Based Motion Deblurring
TLDR
This paper starts from a sequential formulation of event-based motion deblurring, then shows how its optimization can be unfolded with a novel end-toend deep architecture, and proposes a differentiable directional event filtering module to effectively extract rich boundary prior from the evolution of events.
Affine-modeled video extraction from a single motion blurred image
TLDR
A generalized video extraction method using the affine motion modeling, enabling to tackle multiple types of complex motion and their mixing, and validate the state-of-the-art performance of the reported technique.
...
...

References

SHOWING 1-10 OF 44 REFERENCES
Segmentation-Free Dynamic Scene Deblurring
TLDR
This paper proposes a new energy model simultaneously estimating motion flow and the latent image based on robust total variation (TV)-L1 model, and addresses the problem of the traditional coarse-to-fine deblurring framework, which gives rise to artifacts when restoring small structures with distinct motion.
Dynamic Scene Deblurring
TLDR
This paper proposes a novel energy model designed with the weighted sum of multiple blur data models, which estimates different motion blurs and their associated pixel-wise weights, and resulting sharp image and demonstrates that this method outperforms conventional approaches in deblurring both dynamic scenes and static scenes.
Generalized video deblurring for dynamic scenes
TLDR
This work proposes a single energy model that simultaneously estimates optical flows and latent frames to solve the deblurring problem and achieves significant improvements in removing blurs and estimating accurate optical flows in blurry frames.
Modeling Blurred Video with Layers
TLDR
This work develops a novel layered model of scenes in motion that estimates the layer segmentation and each layer’s appearance and motion from a motion-blurred video sequence, and solves the optimization problem as minimizing the pixel error between the blurred frames and images synthesized from the model.
Video Deblurring Algorithm Using Accurate Blur Kernel Estimation and Residual Deconvolution Based on a Blurred-Unblurred Frame Pair
TLDR
This paper presents a novel motion deblurring algorithm in which a blurred frame can be reconstructed utilizing the high-resolution information of adjacent unblurred frames, and shows that the proposed algorithm provides superiordeblurring results over conventional deblurred algorithms while preserving details and reducing ringing artifacts.
Non-uniform Motion Deblurring for Bilayer Scenes
TLDR
The deblurred image of the scene is finally estimated within a regularization framework by accounting for blur variations due to camera motion as well as depth.
A Variational Framework for Simultaneous Motion Estimation and Restoration of Motion-Blurred Video
The problem of motion estimation and restoration of objects in a blurred video sequence is addressed in this paper. Fast movement of the objects, together with the aperture time of the camera, result
Video Segmentation via Object Flow
TLDR
This work forms a principled, multiscale, spatio-temporal objective function that uses optical flow to propagate information between frames for video segmentation and demonstrates the effectiveness of jointly optimizing optical flow and video segmentations using an iterative scheme.
Blind motion deblurring using multiple images
Hand-Held Video Deblurring Via Efficient Fourier Aggregation
TLDR
Experiments with numerous videos recorded in the wild, along with extensive comparisons, show that the proposed algorithm achieves state-of-the-art results while at the same time being much faster than its competitors.
...
...