Time-Multiplexed Coded Aperture Imaging: Learned Coded Aperture and Pixel Exposures for Compressive Imaging Systems

  title={Time-Multiplexed Coded Aperture Imaging: Learned Coded Aperture and Pixel Exposures for Compressive Imaging Systems},
  author={Edwin Vargas and Julien N. P. Martel and Gordon Wetzstein and Henry Arguello},
  journal={2021 IEEE/CVF International Conference on Computer Vision (ICCV)},
Compressive imaging using coded apertures (CA) is a powerful technique that can be used to recover depth, light fields, hyperspectral images and other quantities from a single snapshot. The performance of compressive imaging systems based on CAs mostly depends on two factors: the properties of the mask's attenuation pattern, that we refer to as "codification", and the computational techniques used to recover the quantity of interest from the coded snapshot. In this work, we introduce the idea… 

Figures and Tables from this paper

Acquiring a Dynamic Light Field through a Single-Shot Coded Image

This method is the first to achieve a finer temporal resolution than the camera itself in compressive light-field acquisition and designed an imaging model that synchronously applies aperture coding and pixel-wise exposure coding within a single exposure time.

Deep Optical Coding Design in Computational Imaging

This work surveys the recent advances on CE data-driven design and provides guidelines on how to parametrize different optical elements to include them in the E2E framework, and presents low-level tasks such as spectral imaging reconstruction or high-level functions such as pose estimation with privacy preserving enhanced by using optimal task-based optical architectures.

Learning Spatially Varying Pixel Exposures for Motion Deblurring

Computationally removing the motion blur introduced by camera shake or object motion in a captured image remains a challenging task in computational photography. Deblurring methods are often limited

Learning rich optical embeddings for privacy-preserving lensless image classification

This work casts the optics as an encoder that produces learned embeddings directly at the camera sensor in the context of image classification, where they jointly optimize the encoder’s parameters and those of an image classifier in an end-to-end fashion.

Image sensing with multilayer, nonlinear optical neural networks

Tianyu Wang,1, ∗ Mandar M. Sohoni,1, ∗ Logan G. Wright,1, 2 Martin M. Stein,1 Shi-Yuan Ma,1 Tatsuhiro Onodera,1, 2 Maxwell Anderson,1 and Peter L. McMahon1, 3, † School of Applied and Engineering

Time-Multiplexed Coded Aperture and Coded Focal Stack -Comparative Study on Snapshot Compressive Light Field Imaging

SUMMARY A light field (LF), which is represented as a set of dense, multi-view images, has been used in various 3D applications. To make LF acquisition more e ffi cient, researchers have investigated

Fast Disparity Estimation from a Single Compressed Light Field Measurement

This work proposes a fast disparity estimation from a single compressed measurement by omitting the recovery step required in traditional approaches, and is 20 times faster in training and inference than the best method that estimates the disparity from reconstructed light fields.



Compressive coded aperture imaging

This work proposes compressive imaging techniques for improving the performance of video imaging systems in the presence of constraints on the focal plane array size that combines coded aperture imaging to enhance pixel resolution with superimposing subframes of a scene onto a single focal plane arrays to increase field of view.

Coded Two-Bucket Cameras for Computer Vision

This work introduces coded two-bucket (C2B) imaging, a new operating principle for computational sensors with applications in active 3D shape estimation and coded-exposure imaging, and presents the first experimental demonstration of these capabilities, using a fully-functional C2B camera prototype.

Compressive Light Field Sensing

The proposed acquisition and recovery method provides light field images with high spatial resolution and signal-to-noise-ratio, and therefore is not affected by limitations common to existing light field camera designs.

Compressive Coded Aperture Spectral Imaging: An Introduction

The remarkable advantage of CASSI is that the entire data cube is sensed with just a few FPA measurements and, in some cases, with as little as a single FPA shot.

Adaptive dynamic range imaging: optical control of pixel exposures over space and time

A video-rate adaptive dynamic range camera that consists of a color CCD detector and a controllable liquid crystal light modulator is implemented and indicates that adaptive imaging can have a significant impact on vision applications such as monitoring, tracking, recognition, and navigation.

Lensless Imaging with a Controllable Aperture

  • A. ZometS. Nayar
  • Physics
    2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06)
  • 2006
A novel, highly flexible camera that can be used as a computational sensor, where the detector measures the end result of computations performed by the attenuating layers on the scene radiance values and the merits and limitations of lensless imaging using controllable apertures are discussed.

Programmable aperture photography: multiplexed light field acquisition

A system including a novel component called programmable aperture and two associated post-processing algorithms for high-quality light field acquisition and the effectiveness of the system and the quality of the captured light field is presented.

Colored Coded Aperture Design by Concentration of Measure in Compressive Spectral Imaging

The colored coded apertures are optimized such that the number of projections is minimized while the quality of reconstruction is maximized and better satisfy the restricted isometry property in CASSI.

End-to-end optimization of optics and image processing for achromatic extended depth of field and super-resolution imaging

A fully-differentiable simulation model is built that maps the true source image to the reconstructed one and jointly optimize the optical parameters and the image processing algorithm parameters so as to minimize the deviation between the true and reconstructed image, over a large set of images.

Snapshot colored compressive spectral imager.

A compact snapshot colored compressive spectral imager (SCCSI) that exploits the benefits of the colored mosaic FPA detectors and the compression capabilities of CSI sensing techniques and can capture the spatiospectral information of a scene in a single snapshot.