Combining LiDAR space clustering and convolutional neural networks for pedestrian detection

  title={Combining LiDAR space clustering and convolutional neural networks for pedestrian detection},
  author={Damien Matti and Hazim Kemal Ekenel and Jean-Philippe Thiran},
  journal={2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS)},
  • Damien Matti, H. K. Ekenel, J. Thiran
  • Published 1 August 2017
  • Computer Science
  • 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS)
Pedestrian detection is an important component for safety of autonomous vehicles, as well as for traffic and street surveillance. [] Key Method In the proposed approach, LiDAR data is utilized to generate region proposals by processing the three dimensional point cloud that it provides. These candidate regions are then further processed by a state-of-the-art CNN classifier that we have fine-tuned for pedestrian detection. We have extensively evaluated the proposed detection process on the KITTI dataset. The…

Figures and Tables from this paper

Multimodal CNN Pedestrian Classification: A Study on Combining LIDAR and Camera Data

This paper presents a study on pedestrian classification based on deep learning using data from a monocular camera and a 3D LIDAR sensor, separately and in combination. Early and late multi-modal

Pedestrian Detection in 3D Point Clouds using Deep Neural Networks

The hypothesis that 3D geometric information is essential for a neural network to learn to detect pedestrians in outdoor scenes, thus proving that LIDAR sensors should be a must in these systems is confirmed.

A Pedestrian Detection and Tracking Framework for Autonomous Cars: Efficient Fusion of Camera and LiDAR Data

A novel method for pedestrian detection and tracking by fusing camera and LiDAR sensor data and a deep neural network is developed to identify pedestrian candidates both in RGB and depth images.

Mahir Gulzar Object Detection Using LiDAR and Camera Fusion in Off-road Conditions

This thesis presents a fusion based approach for detecting objects in 3D by projecting the proposed 2D regions of interest or masks to point clouds and applies outlier filtering techniques to filter out target object points in projected regions ofinterest.

Deep Multi-Modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges

This review paper attempts to systematically summarize methodologies and discuss challenges for deep multi-modal object detection and semantic segmentation in autonomous driving with an overview of on-board sensors on test vehicles, open datasets, and background information.

Multi-Sensor Fusion Perception System in Train

This paper proposes a detection and ranging fusion method based on one Lidar and two different focal length cameras, which be applied in the subway system, to alert drivers to possible obstacles and assist brake.

A Method of Pedestrian Fine-grained Attribute Detection and Recognition

A method that employs the fusion of convolutional neural network models based on multi-task learning for multi-attributes to solve the problem of a low degree of accuracy in detecting and recognizing pedestrians fine-grained attributes under complex circumstances.

Object Detection for Autonomous Vehicle with LiDAR Using Deep Learning

The study attempts to employ the Light Detection and Ranging (LiDAR) sensor that uses light in the form of a pulsed laser to calculate ranges and ultimately detect objects and shows that YOLOv2 can identify the objects better compared to Single Shot Detection (SSD) algorithm.



Robust detection of non-motorized road users using deep learning on optical and LIDAR data

  • Taewan KimJoydeep Ghosh
  • Computer Science
    2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC)
  • 2016
This paper proposes a fusion of LIDAR data and a deep learning-based computer vision algorithm, to substantially improve the detection of regions of interest (ROIs) and subsequent identification of road users.

Pedestrian detection combining RGB and dense LIDAR data

A state-of-the-art deformable parts detector is trained using different configurations of optical images and their associated 3D point clouds, in conjunction and independently, leveraging upon the recently released KITTI dataset to propose novel strategies for depth upsampling and contextual fusion that together lead to detection performance which exceeds that of the RGB-only systems.

Object Detection and Classification by Decision-Level Fusion for Intelligent Vehicle Systems

This paper proposes a new object-detection and classification method using decision-level fusion that fuse the classification outputs from independent unary classifiers, such as 3D point clouds and image data using a convolutional neural network (CNN).

Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation

This paper proposes a simple and scalable detection algorithm that improves mean average precision (mAP) by more than 30% relative to the previous best result on VOC 2012 -- achieving a mAP of 53.3%.

Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks

This work introduces a Region Proposal Network (RPN) that shares full-image convolutional features with the detection network, thus enabling nearly cost-free region proposals and further merge RPN and Fast R-CNN into a single network by sharing their convolutionAL features.

Multimodal People Detection and Tracking in Crowded Scenes

A novel people detection and tracking method based on a multi-modal sensor fusion approach that utilizes 2D laser range and camera data and consists in a fast and detailed analysis of the spatial distribution of voters per detected person.

Pedestrian Detection and Tracking in an Urban Environment Using a Multilayer Laser Scanner

To improve the robustness of pedestrian detection using a single laser sensor, a detection system based on the fusion of information located in the four laser planes is proposed, which uses a nonparametric kernel-density-based estimation of pedestrian position of each laser plane.

Ten Years of Pedestrian Detection, What Have We Learned?

This work analyzes the remarkable progress of the last decade by dis- cussing the main ideas explored in the 40+ detectors currently present in the Caltech pedestrian detection benchmark to find a new decision forest detector.

Are we ready for autonomous driving? The KITTI vision benchmark suite

The autonomous driving platform is used to develop novel challenging benchmarks for the tasks of stereo, optical flow, visual odometry/SLAM and 3D object detection, revealing that methods ranking high on established datasets such as Middlebury perform below average when being moved outside the laboratory to the real world.

3D Object Proposals for Accurate Object Class Detection

This method exploits stereo imagery to place proposals in the form of 3D bounding boxes in the context of autonomous driving and outperforms all existing results on all three KITTI object classes.