The Frankencamera: an experimental platform for computational photography

@article{Adams2012TheFA,
  title={The Frankencamera: an experimental platform for computational photography},
  author={Andrew Adams and David E. Jacobs and Jennifer Dolson and Marius Tico and Kari Pulli and Eino-Ville Talvala and Boris Ajdin and Daniel Andr{\'e} Vaquero and Hendrik P. A. Lensch and Mark Horowitz and Sung Hee Park and Natasha Gelfand and Jongmin Baek and Wojciech Matusik and Marc Levoy},
  journal={ACM Trans. Graph.},
  year={2012},
  volume={29},
  pages={29:1-29:12}
}
Although there has been much interest in computational photography within the research and photography communities, progress has been hampered by the lack of a portable, programmable camera with sufficient image quality and computing power. To address this problem, we have designed and implemented an open architecture and application programming interface (API) for such cameras: the Frankencamera. It consists of a base hardware specification, a software stack based on Linux, and an API for C… 
View on ACM
cs.cmu.edu
97 Citations
Highly Influential Citations
6
Background Citations
25
Methods Citations
22

Figures from this paper

97 Citations

Citation Type

Citation Type

Computational Cameras: Approaches, Benefits and Limits
TLDR
The benefits and limits of computational imaging are explored, and how it is related to the adjacent and overlapping fields of digital imaging, computational photography and computational image sensors is discussed.
Computational Cameras: Convergence of Optics and Processing
TLDR
This paper uses the abstract notion of light field representation as a general tool to describe computational camera designs, where each camera can be formulated as a projection of a high-dimensional light field to a 2-D image sensor.
Noise and dynamic range optimal computational imaging
TLDR
This paper proposes a novel computational imaging system that automatically and efficiently computes the optimal number of shots and corresponding exposure times and gains, taking into account characteristics of the scene and sensor.
FlexISP: a flexible camera image processing framework
  • K. Pulli
  • Computer Science
    ACM Trans. Graph.
  • 2014
TLDR
This work proposes an end-to-end system that is aware of the camera and image model, enforces natural-image priors, while jointly accounting for common image processing steps like demosaicking, denoising, deconvolution, and so forth, all directly in a given output representation.
FCam for multiple cameras
TLDR
An extension to FCam API is presented that allows for an enumeration of cameras and their corresponding properties, such as position or orientation, and explicitly support camera synchronization, either through hardware mechanisms or software primitives.
The FCam API for programmable cameras
TLDR
The architecture underlying the FCam API, the design of the API itself, several applications built on top of it, and some examples of its use in education are described.
Burst photography for high dynamic range and low-light imaging on mobile cameras
TLDR
A computational photography pipeline that captures, aligns, and merges a burst of frames to reduce noise and increase dynamic range, built atop Android's Camera2 API and written in the Halide domain-specific language (DSL).
Mobile Computational Photography with FCam
TLDR
This chapter describes how FCam has been used at many universities to teach computational photography and several example programs that run on the NVIDIA Tegra 3 prototype tablet and the Nokia N900 and N9 Linux-based phones.
Freehand HDR Imaging of Moving Scenes with Simultaneous Resolution Enhancement
TLDR
The alignment step is performed by a modern energy‐based optic flow approach that takes into account the varying exposure conditions, and produces dense displacement fields with subpixel precision, which can handle arbitrary complex motion patterns, caused by severe camera shake and moving objects.
Rigel: flexible multi-rate image processing hardware
TLDR
This paper presents Rigel, which takes pipelines specified in the new multi-rate architecture and lowers them to FPGA implementations, and demonstrates depth from stereo, Lucas-Kanade, the SIFT descriptor, and a Gaussian pyramid running on two FPGAs.
...
...

References

SHOWING 1-10 OF 50 REFERENCES
Viewfinder Alignment
TLDR
An algorithm for aligning successive viewfinder frames is described, which is robust to noise, never requires storing more than one viewfinder frame in memory, and runs at 30 frames per second on standard smartphone hardware.
High performance imaging using large camera arrays
TLDR
A unique array of 100 custom video cameras that are built are described, and their experiences using this array in a range of imaging applications are summarized.
Image deblurring using inertial measurement sensors
TLDR
This work uses a combination of inexpensive gyroscopes and accelerometers in an energy optimization framework to estimate a blur function from the camera's acceleration and angular velocity during an Exposure to solve for the camera motion at a high sampling rate during an exposure and infer the latent image using a joint optimization.
CMUcam3: An Open Programmable Embedded Vision Sensor
TLDR
TheCMUcam3 is the third generation of the CMUcam system and is designed to provide a flexible and easy to use open source development environment along with a more powerful hardware platform.
High Dynamic Range Imaging: Acquisition, Display, and Image-Based Lighting (The Morgan Kaufmann Series in Computer Graphics)
TLDR
This landmark book is the first to describe HDRI technology in its entirety and covers a wide-range of topics, from capture devices to tone reproduction and image-based lighting, leading to an unparalleled visual experience.
Computational Re-Photography
TLDR
This work presents a real-time estimation and visualization technique for rephotography that helps users reach a desired viewpoint during capture and demonstrates the success of the technique by rephotographing historical images and conducting user studies.
Computational Photography: Mastering New Techniques for Lenses, Lighting, and Sensors
TLDR
The computational techniques discussed cover topics in exploiting new ideas in manipulating optics, illumination, and sensors at time of capture and reconstruction procedures that go well beyond the traditional digital darkroom experience.
Being `undigital' with digital cameras: extending dynamic range by combining differently exposed pictures
TLDR
This work proposes a means of combining diierently exposed pictures to obtain a single picture of extended dynamic range, and improved color delity, which works reliably with images from a digital camera of unknown response, or from a scanner with unknown response.
Smart Cameras as Embedded Systems
Recent technological advances are enabling a new generation of smart cameras that represent a quantum leap in sophistication. While today's digital cameras capture images, smart cameras capture…
MeshEye: A Hybrid-Resolution Smart Camera Mote for Applications in Distributed Intelligent Surveillance
TLDR
MeshEye is introduced, an energy-efficient smart camera mote architecture that has been designed with intelligent surveillance as the target application in mind and basic vision algorithms for object detection, acquisition, and tracking are described and illustrated on real- world data.
...
...