SUNet: Symmetric Undistortion Network for Rolling Shutter Correction

@article{Fan2021SUNetSU,
  title={SUNet: Symmetric Undistortion Network for Rolling Shutter Correction},
  author={Bin Fan and Yuchao Dai and Mingyi He},
  journal={2021 IEEE/CVF International Conference on Computer Vision (ICCV)},
  year={2021},
  pages={4521-4530}
}
  • Bin Fan, Yuchao Dai, Mingyi He
  • Published 10 August 2021
  • Computer Science
  • 2021 IEEE/CVF International Conference on Computer Vision (ICCV)
The vast majority of modern consumer-grade cameras employ a rolling shutter mechanism, leading to image distortions if the camera moves during image acquisition. In this paper, we present a novel deep network to solve the generic rolling shutter correction problem with two consecutive frames. Our pipeline is symmetrically designed to predict the global shutter image corresponding to the intermediate time of these two frames, which is difficult for existing methods because it corresponds to a… 
Inverting a Rolling Shutter Camera: Bring Rolling Shutter Images to High Framerate Global Shutter Video
  • Bin Fan, Yuchao Dai
  • Computer Science
    2021 IEEE/CVF International Conference on Computer Vision (ICCV)
  • 2021
TLDR
The first RS temporal super-resolution network in a cascaded structure to extract high framerate global shutter video and explores the underlying spatio-temporal geometric relationships within a deep learning framework, where no extra supervision besides the middle-scanline ground truth GS image is needed.
Learning Adaptive Warping for Real-World Rolling Shutter Correction
TLDR
This paper proposes the first real-world rolling shutter (RS) correction dataset, BS-RSC, and a corresponding model to correct the RS frames in a distorted video, and can improve the model’s ability to remove the RS effects in the real world.
Bringing Rolling Shutter Images Alive with Dual Reversed Distortion
TLDR
A novel end-to-end model is developed, IFED, to generate dual optical flow sequence through iterative learning of the velocity field during the RS time, which is superior to naive cascade schemes, as well as the state-of-the-art which utilizes adjacent RS images.
Rolling-Shutter-stereo-aware motion estimation and image correction

References

SHOWING 1-10 OF 38 REFERENCES
Rolling Shutter Correction in Manhattan World
TLDR
This paper proposes a novel method which leverages geometric properties of the scene—in particular vanishing directions—to estimate the camera motion during rolling shutter exposure from a single distorted image.
Occlusion-Aware Rolling Shutter Rectification of 3D Scenes
TLDR
A first-of-its-kind pipeline to recover the latent image of a 3D scene from a set of such RS distorted images by sequentially recovering both the camera motion and scene structure while accounting for RS and occlusion effects is developed.
Inverting a Rolling Shutter Camera: Bring Rolling Shutter Images to High Framerate Global Shutter Video
  • Bin Fan, Yuchao Dai
  • Computer Science
    2021 IEEE/CVF International Conference on Computer Vision (ICCV)
  • 2021
TLDR
The first RS temporal super-resolution network in a cascaded structure to extract high framerate global shutter video and explores the underlying spatio-temporal geometric relationships within a deep learning framework, where no extra supervision besides the middle-scanline ground truth GS image is needed.
From Two Rolling Shutters to One Global Shutter
TLDR
This work explores a surprisingly simple camera configuration that makes it possible to undo the rolling shutter distortion: two cameras mounted to have different rolling shutter directions.
Deep Shutter Unrolling Network
TLDR
A novel network for rolling shutter effect correction that can be trained end-to-end and only requires the global shutter image for supervision is presented and experimental results demonstrate that it outperforms the state-of-the-art methods.
Rolling Shutter Camera Relative Pose: Generalized Epipolar Geometry
TLDR
The rolling shutter essential matrix is introduced, and its link to existing models such as the push-broom cameras is established, summarized in a clean hierarchy of multi-perspective cameras.
Rolling-Shutter-Aware Differential SfM and Image Rectification
TLDR
A modified differential Structure from Motion (SfM) algorithm that can estimate relative pose from two consecutive frames despite of Rolling Shutter (RS) artifacts is developed and outperforms state-of-the-art commercial software products, i.e. Adobe After Effects and Apple Imovie, at removing RS artifacts.
Image Stitching and Rectification for Hand-Held Cameras
TLDR
A new differential homography that can account for the scanline-varying camera poses in Rolling Shutter (RS) cameras is derived, and its application to carry out RS-aware image stitching and rectification at one stroke is demonstrated.
Learning Structure-And-Motion-Aware Rolling Shutter Correction
TLDR
This paper first makes a theoretical contribution by showing that RS two-view geometry is degenerate in the case of pure translational camera motion, and proposes a Convolutional Neural Network (CNN)-based method which learns the underlying geometry from just a single RS image and performs RS image correction.
From Bows to Arrows: Rolling Shutter Rectification of Urban Scenes
TLDR
A procedure to extract prominent curves from the RS image since this is essential for deciphering the varying row-wise motion and an optimization problem with line desirability costs based on straightness, angle, and length to resolve the geometric ambiguities.
...
1
2
3
4
...