Event-Based Vision: A Survey

@article{Gallego2022EventBasedVA,
  title={Event-Based Vision: A Survey},
  author={Guillermo Gallego and Tobi Delbruck and G. Orchard and Chiara Bartolozzi and Brian Taba and Andrea Censi and Stefan Leutenegger and Andrew J. Davison and J{\"o}rg Conradt and Kostas Daniilidis and Davide Scaramuzza},
  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year={2022},
  volume={44},
  pages={154-180}
}
Event cameras are bio-inspired sensors that differ from conventional frame cameras: Instead of capturing images at a fixed rate, they asynchronously measure per-pixel brightness changes, and output a stream of events that encode the time, location and sign of the brightness changes. Event cameras offer attractive properties compared to traditional cameras: high temporal resolution (in the order of $\mu$μs), very high dynamic range (140 dB versus 60 dB), low power consumption, and high pixel… 
Stereo dense depth tracking based on optical flow using frames and events
TLDR
This work proposes to estimate dense disparity from standard frames at the point of their availability, predict the disparity using odometry information, and track the disparity asynchronously using optical flow of events between the standard frames.
Globally-Optimal Event Camera Motion Estimation
TLDR
The present paper looks at fronto-parallel motion estimation of an event camera, and derives a globally optimal solution to this generally non-convex problem, and removes the dependency on a good initial guess.
Towards Real-Time Edge Detection for Event Cameras Based on Lifetime and Dynamic Slicing
TLDR
An algorithm is developed to extract edges from events by augmenting a batch of events with their lifetimes by using a batching technique to increase the frame rate of generated images since events with a high sample rate cause the processing of a single event to be computationally expensive.
Event Enhanced High-Quality Image Recovery
TLDR
An explainable network, an event-enhanced sparse learning network (eSL-Net), to recover the high-quality images from event cameras and can largely improve the performance of the state-of-the-art by 7-12 dB.
Event-Based Pedestrian Detection Using Dynamic Vision Sensors
TLDR
A novel event-to-frame conversion method by integrating the inherent characteristics of events more efficiently is proposed and an improved feature extraction network that can reuse intermediate features to further reduce the computational effort is designed.
Event Camera Simulator Improvements via Characterized Parameters
TLDR
This work presents an extended DVS pixel simulator for neuromorphic benchmarks which simplifies the latency and the noise models, and to more closely model the behaviour of a real pixel, the readout circuitry is modelled.
Event-based tracking of human hands
TLDR
The proposed ROI-finding method reduces noise from intensity images, achieving up to 89% of data reduction in relation to the original, while preserving the features of the region of interest (ROI).
Hybrid SNN-ANN: Energy-Efficient Classification and Object Detection for Event-Based Vision
TLDR
This article proposes a hybrid architecture for end-to-end training of deep neural networks for event-based pattern recognition and object detection, combining a spiking neural network backbone for efficient event- based feature extraction, and a subsequent classical analog neural network (ANN) head to solve synchronous classification and detection tasks.
Asynchronous time-based imager with DVS sharing
TLDR
The Verilog-A model simulation of a 4 $$\times$$ 4 matrix is 70% faster than the electrical device-level simulation, while yielding similar reconstruction results.
Event-Based Robotic Grasping Detection With Neuromorphic Vision Sensor and Event-Grasping Dataset
TLDR
A deep neural network for grasping detection is developed that considers the angle learning problem as classification instead of regression, and performs high detection accuracy on the Event-Grasping dataset with 93% precision at an object-wise level split.
...
1
2
3
4
...

References

SHOWING 1-10 OF 262 REFERENCES
Event-Based, 6-DOF Camera Tracking from Photometric Depth Maps
TLDR
This paper tackles the problem of accurate, low-latency tracking of an event camera from an existing photometric depth map built via classic dense reconstruction pipelines, and tracks the 6-DOF pose of the event camera upon the arrival of each event, thus virtually eliminating latency.
CED: Color Event Camera Dataset
TLDR
This work presents and releases the first Color Event Camera Dataset (CED), containing 50 minutes of footage with both color frames and events, and presents an extension of the event camera simulator ESIM that enables simulation of color events.
Focus Is All You Need: Loss Functions for Event-Based Vision
TLDR
This work presents a collection and taxonomy of twenty two objective functions to analyze event alignment in motion compensation approaches, and concludes that the variance, the gradient and the Laplacian magnitudes are among the best loss functions.
The Multivehicle Stereo Event Camera Dataset: An Event Camera Dataset for 3D Perception
TLDR
This letter presents a large dataset with a synchronized stereo pair event based camera system, carried on a handheld rig, flown by a hexacopter, driven on top of a car, and mounted on a motorcycle, in a variety of different illumination levels and environments.
Lifetime estimation of events from Dynamic Vision Sensors
TLDR
An algorithm is developed that augments each event with its lifetime, which is computed from the event's velocity on the image plane, which gives a continuous representation of events in time, hence enabling the design of new algorithms that outperform those based on the accumulation of events over fixed, artificially-chosen time intervals.
EMVS: Event-Based Multi-View Stereo—3D Reconstruction with an Event Camera in Real-Time
TLDR
This work introduces the problem of event-based multi-view stereo (EMVS) for event cameras and proposes a solution that elegantly exploits two inherent properties of an event camera: its ability to respond to scene edges—which naturally provide semi-dense geometric information without any pre-processing operation— and the fact that it provides continuous measurements as the sensor moves.
Simultaneous Optical Flow and Intensity Estimation from an Event Camera
TLDR
This work proposes, to the best of the knowledge, the first algorithm to simultaneously recover the motion field and brightness image, while the camera undergoes a generic motion through any scene, within a sliding window time interval.
Fast Event-based Corner Detection
TLDR
This work proposes a method to reduce an event stream to a corner event stream, which is capable of pro- cessing millions of events per second on a single core and reduces the event rate by a factor of 10 to 20.
High Speed and High Dynamic Range Video with an Event Camera
TLDR
This work proposes a novel recurrent network to reconstruct videos from a stream of events, and trains it on a large amount of simulated event data, and shows that off-the-shelf computer vision algorithms can be applied to the reconstructions and that this strategy consistently outperforms algorithms that were specifically designed for event data.
Event-based, Direct Camera Tracking from a Photometric 3D Map using Nonlinear Optimization
TLDR
This work presents a method to track the 6-DOF pose of an event camera in a known environment, which is contemplated to be described by a photometric 3D map built via classic dense 3D reconstruction algorithms.
...
1
2
3
4
5
...