Anomaly Detection in Autonomous Driving: A Survey

  title={Anomaly Detection in Autonomous Driving: A Survey},
  author={Daniel Bogdoll and Maximilian Nitsche and Johann Marius Z{\"o}llner},
Nowadays, there are outstanding strides towards a future with autonomous vehicles on our roads. While the perception of autonomous vehicles performs well under closed-set conditions, they still struggle to handle the unexpected. This survey provides an extensive overview of anomaly detection techniques based on camera, lidar, radar, multimodal and abstract object level data. We provide a systematization including detection approach, corner case level, ability for an online application, and… 

Figures and Tables from this paper

Multimodal Detection of Unknown Objects on Roads for Autonomous Driving
A novel pipeline to detect unknown objects is proposed that makes use of lidar and camera data by combining state-of-the art detection models in a sequential manner and evaluates the approach on the Waymo Open Perception Dataset and points out current research gaps in anomaly detection.
Experiments on Anomaly Detection in Autonomous Driving by Forward-Backward Style Transfers
A novel approach for anomaly detection through style transfer is proposed, which leverages generative models to map an image from its original style domain of road traffic to an arbitrary one and back to generate pixelwise anomaly scores.


nuScenes: A Multimodal Dataset for Autonomous Driving
Robust detection and tracking of objects is crucial for the deployment of autonomous vehicle technology. Image based benchmark datasets have driven development in computer vision tasks such as object
Identification of Ghost Moving Detections in Automotive Scenarios with Deep Learning
A fully connected network is used to distinguish between real and false moving detections in the occupancy gridmaps by using the local Doppler information, along with the spatial context of the surrounding scenario to classify themoving detections.
Road Anomaly Detection by Partial Image Reconstruction with Segmentation Coupling
A reconstruction module that can be used with many existing semantic segmentation networks, and that is trained to recognize and reconstruct road (drivable) surface from a small bottleneck, is proposed, postulate that poor reconstruction of the road surface is due to areas that are outside of the training distribution, which is a strong indicator of an anomaly.
CODA: A Real-World Road Corner Case Dataset for Object Detection in Autonomous Driving
A challenging dataset named CODA, which consists of 1500 carefully selected real-world driving scenes, each containing four object-level corner cases (on aver-age), spanning more than 30 object categories, is introduced that exposes this critical problem of vision-based detectors.
Real-time small obstacle detection on highways using compressive RBM road reconstruction
A technique to detect anomalous patches on the road from color images using a Restricted Boltzman Machine neural network specifically trained to reconstruct the appearance of the road yields a more relevant segmentation of anomalies than classic image processing techniques.
Lost and Found: detecting small road hazards for self-driving vehicles
The proposed algorithm performs statistical hypothesis tests in disparity space directly on stereo image data, assessing freespace and obstacle hypotheses on independent local patches and outperforms all considered baselines in evaluations on both pixel and object level.
Towards Corner Case Detection for Autonomous Driving
This paper provides a formal definition of a corner case and proposes a system framework for both the online and the offline use case that can handle video signals from front cameras of a naturally moving vehicle and can output a corners case score.
Detection of Collective Anomalies in Images for Automated Driving Using an Earth Mover’s Deviation (EMDEV) Measure
This work proposes a detection method for collective anomalies based on a comparison of a test (sub-)set instance distribution to a training (i.e., reference) instance distribution, both distributions obtained by an instance-based semantic segmentation.
Detecting Road Obstacles by Erasing Them
This work selects image patches and inpaints them with the surrounding road texture, which tends to remove obstacles from those patches, and uses a network trained to recognize discrepancies between the original patch and the inpainted one, which signals an erased obstacle.
Systematization of Corner Cases for Visual Perception in Automated Driving
This paper provides a systematization of corner cases for visual perception in automated driving, with the categories being structured by detection complexity.