Physics-based Noise Modeling for Extreme Low-light Photography

  title={Physics-based Noise Modeling for Extreme Low-light Photography},
  author={Kaixuan Wei and Ying Fu and Yinqiang Zheng and Jiaolong Yang},
  journal={IEEE transactions on pattern analysis and machine intelligence},
  • Kaixuan Wei, Ying Fu, +1 author Jiaolong Yang
  • Published 2021
  • Medicine, Computer Science, Engineering
  • IEEE transactions on pattern analysis and machine intelligence
Enhancing the visibility in extreme low-light environments is a challenging task. Under nearly lightless condition, existing image denoising methods could easily break down due to significantly low SNR. In this paper, we systematically study the noise statistics in the imaging pipeline of CMOS photosensors, and formulate a comprehensive noise model that can accurately characterize the real noise structures. Our novel model considers the noise sources caused by digital camera electronics which… Expand


A Physics-Based Noise Formation Model for Extreme Low-Light Raw Denoising
A highly accurate noise formation model based on the characteristics of CMOS photosensors is presented, thereby enabling us to synthesize realistic samples that better match the physics of image formation process. Expand
Learning to See in the Dark
A pipeline for processing low-light images is developed, based on end-to-end training of a fully-convolutional network that operates directly on raw sensor data and replaces much of the traditional image processing pipeline, which tends to perform poorly on such data. Expand
Enhancing Low Light Videos by Exploring High Sensitivity Camera Noise
The physical origins of the practical high sensitivity noise in digital cameras are explored, model them mathematically, and propose to enhance the low light videos based on the noise model by using an LSTM-based neural network. Expand
Toward Convolutional Blind Denoising of Real Photographs
A convolutional blind denoising network (CBDNet) with more realistic noise model and real-world noisy-clean image pairs and a noise estimation subnetwork with asymmetric learning to suppress under-estimation of noise level is embedded into CBDNet. Expand
Seeing Motion in the Dark
By carefully designing a learning-based pipeline and introducing a new loss function to encourage temporal stability, a siamese network is trained on static raw videos, for which ground truth is available, such that the network generalizes to videos of dynamic scenes at test time. Expand
Deep joint demosaicking and denoising
A new data-driven approach forDemosaicking and denoising is introduced: a deep neural network is trained on a large corpus of images instead of using hand-tuned filters and this network and training procedure outperform state-of-the-art both on noisy and noise-free data. Expand
Noise Flow: Noise Modeling With Conditional Normalizing Flows
Noise Flow is introduced, a powerful and accurate noise model based on recent normalizing flow architectures that represents the first serious attempt to go beyond simple parametric models to one that leverages the power of deep learning and data-driven noise distributions. Expand
Unprocessing Images for Learned Raw Denoising
This work presents a technique to “unprocess” images by inverting each step of an image processing pipeline, thereby allowing us to synthesize realistic raw sensor measurements from commonly available Internet photos. Expand
Benchmarking Denoising Algorithms with Real Photographs
  • Tobias Plötz, S. Roth
  • Computer Science
  • 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2017
This work develops a methodology for benchmarking denoising techniques on real photographs, and captures pairs of images with different ISO values and appropriately adjusted exposure times, where the nearly noise-free low-ISO image serves as reference. Expand
A Trilateral Weighted Sparse Coding Scheme for Real-World Image Denoising
This paper develops a trilateral weighted sparse coding (TWSC) scheme for robust real-world image denoising and introduces three weight matrices into the data and regularization terms of the sparse coding framework to characterize the statistics of realistic noise and image priors. Expand