Making Deep Heatmaps Robust to Partial Occlusions for 3D Object Pose Estimation

  title={Making Deep Heatmaps Robust to Partial Occlusions for 3D Object Pose Estimation},
  author={Markus Oberweger and Mahdi Rad and Vincent Lepetit},
We introduce a novel method for robust and accurate 3D object pose estimation from a single color image under large occlusions. [] Key Method Unfortunately, as the results of our experiments show, predicting these 2D projections using a regular CNN or a Convolutional Pose Machine is highly sensitive to partial occlusions, even when these methods are trained with partially occluded examples.
Occlusion-Robust Object Pose Estimation with Holistic Representation
A novel occlude-and-blackout batch augmentation technique to learn occlusion-robust deep features, and a multi-precision supervision architecture to encourage holistic pose representation learning for accurate and coherent landmark predictions are developed.
Segmentation-Driven 6D Object Pose Estimation
This paper introduces a segmentation-driven 6D pose estimation framework where each visible part of the objects contributes a local pose prediction in the form of 2D keypoint locations and uses a predicted measure of confidence to combine these pose candidates into a robust set of 3D-to-2D correspondences.
Pix2Pose: Pixel-Wise Coordinate Regression of Objects for 6D Pose Estimation
A novel pose estimation method that predicts the 3D coordinates of each object pixel without textured models, and a novel loss function, the transformer loss, is proposed to handle symmetric objects by guiding predictions to the closest symmetric pose.
ZebraPose: Coarse to Fine Surface Encoding for 6DoF Object Pose Estimation
This work presents a discrete descriptor, which can represent the object surface densely by incorporating a hierarchical binary grouping, and proposes a coarse to coarse training strategy, which enables coarse correspondence prediction of the 6DoF pose.
DPOD: 6D Pose Object Detector and Refiner
A novel deep learning method that estimates dense multi-class 2D-3D correspondence maps between an input image and available 3D models and demonstrates that a large number of correspondences is beneficial for obtaining high-quality 6D poses both before and after refinement.
PrimA6D: Rotational Primitive Reconstruction for Enhanced and Robust 6D Pose Estimation
A rotational primitive prediction based 6D object pose estimation using a single image as an input using a Variational AutoEncoder to learn orientation-induced primitive so as to achieve the pose estimation accuracy regardless of the object size.
3D Object Detection and Pose Estimation of Unseen Objects in Color Images with Local Surface Embeddings
This work presents an approach for detecting and estimating the 3D poses of objects in images that requires only an untextured CAD model and no training phase for new objects, and relies on an embedding of local 3D geometry to match the CAD models to the input images.
DCNet: Dense Correspondence Neural Network for 6DoF Object Pose Estimation in Occluded Scenes
6DoF object pose estimation is essential for many real-world applications. Although great progress has been made, challenges still remain in estimating 6D pose for occluded objects. Current RGB-D
3D-Aware Ellipse Prediction for Object-Based Camera Pose Estimation
A learning-based method which detects improved elliptic approximations of objects which are coherent with the 3D ellipsoid in terms of perspective projection and is more robust to the variability of the boundaries of the detection boxes.


BB8: A Scalable, Accurate, Robust to Partial Occlusion Method for Predicting the 3D Poses of Challenging Objects without Using Depth
  • Mahdi Rad, V. Lepetit
  • Computer Science
    2017 IEEE International Conference on Computer Vision (ICCV)
  • 2017
A novel method for 3D object detection and pose estimation from color images only that uses segmentation to detect the objects of interest in 2D even in presence of partial occlusions and cluttered background and is the first to report results on the Occlusion dataset using color imagesonly.
PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes
This work introduces PoseCNN, a new Convolutional Neural Network for 6D object pose estimation, which is highly robust to occlusions, can handle symmetric objects, and provide accurate pose estimation using only color images as input.
iPose: Instance-Aware 6D Pose Estimation of Partly Occluded Objects
This work presents the first deep learning-based system that estimates accurate poses for partly occluded objects from RGB-D and RGB input with a new instance-aware pipeline that decomposes 6D object pose estimation into a sequence of simpler steps, where each step removes specific aspects of the problem.
Real-Time Seamless Single Shot 6D Object Pose Prediction
A single-shot approach for simultaneously detecting an object in an RGB image and predicting its 6D pose without requiring multiple stages or having to examine multiple hypotheses is proposed, which substantially outperforms other recent CNN-based approaches when they are all used without postprocessing.
Learning 6D Object Pose Estimation Using 3D Object Coordinates
This work addresses the problem of estimating the 6D Pose of specific objects from a single RGB-D image by presenting a learned, intermediate representation in form of a dense 3D object coordinate labelling paired with a dense class labelling.
Uncertainty-Driven 6D Pose Estimation of Objects and Scenes from a Single RGB Image
A regularized, auto-context regression framework is developed which iteratively reduces uncertainty in object coordinate and object label predictions and an efficient way to marginalize object coordinate distributions over depth is introduced to deal with missing depth information.
Feature Mapping for Learning Fast and Accurate 3D Pose Inference from Synthetic Images
This paper proposes a simple and efficient method for exploiting synthetic images when training a Deep Network to predict a 3D pose from an image, and shows that it performs very well in practice, and inference is faster and more accurate than with an exemplar-based approach.
Deep Learning of Local RGB-D Patches for 3D Object Detection and 6D Pose Estimation
A 3D object detection method that uses regressed descriptors of locally-sampled RGB-D patches for 6D vote casting that generalizes well to previously unseen input data, and delivers robust detection results that compete with and surpass the state-of-the-art while being scalable in the number of objects is presented.
Robust 3D Object Tracking from Monocular Images Using Stable Parts
An algorithm for estimating the pose of a rigid object in real-time under challenging conditions, using a novel representation for the 3D pose of object parts in the form of the 2D projections of a few control points for practical Augmented Reality applications including industrial environments.
Siamese Regression Networks with Efficient mid-level Feature Extraction for 3D Object Pose Estimation
This paper proposes an end-to-end learning framework for directly regressing object poses by exploiting Siamese Networks, and imposes a similarity measure between the representation of the sample images in the feature and pose space respectively, that is shown to boost regression performance.