Templates for 3D Object Pose Estimation Revisited: Generalization to New Objects and Robustness to Occlusions

@article{Nguyen2022TemplatesF3,
  title={Templates for 3D Object Pose Estimation Revisited: Generalization to New Objects and Robustness to Occlusions},
  author={Van Nguyen Nguyen and Yinlin Hu and Yang Xiao and Mathieu Salzmann and Vincent Lepetit},
  journal={2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2022},
  pages={6761-6770}
}
We present a method that can recognize new objects and estimate their 3D pose in RGB images even under partial occlusions. Our method requires neither a training phase on these objects nor real images depicting them, only their CAD models. It relies on a small set of training objects to learn local object representations, which allow us to locally match the input image to a set of “templates”, rendered images of the CAD models for the new objects. In contrast with the state-of-the-art methods… 

Gen6D: Generalizable Model-Free 6-DoF Object Pose Estimation from RGB Images

Experiments show that Gen6D achieves state-of-the-art results on two model-free datasets: the MOPED dataset and a new GenMOP dataset collected by us, and achieves competitive results compared with instance-specific pose estimators.

Perspective Flow Aggregation for Data-Limited 6D Object Pose Estimation

This paper proposes a method that can be trained solely on synthetic images, or optionally using a few additional real ones, and performs on par with methods that require annotated real images for training when not using any, and outperforms them considerably when using as few as twenty real images.

Correct and Certify: A New Approach to Self-Supervised 3D-Object Perception

The proposed self-supervised training approach achieves performance comparable to fully supervised baselines while not requiring pose or keypoint supervision on real data.

ElectricEye: Metallic Object Pose Estimation

  • Computer Science
  • 2022
This thesis aims at learning to extract edges of metallic objects as latent representation to match these with pre-computed templates for pose retrieval to revisit classical concepts of computer vision and recycling these using recent advancements of machine learning.

PIZZA: A Powerful Image-only Zero-Shot Zero-CAD Approach to 6 DoF Tracking

A method for tracking the 6D motion of objects in RGB video sequences when neither the training images nor the 3D geometry of the objects are available, which can consider unknown objects in open world instantly, without requiring any prior information or a specific training phase.

Template-Based Category-Agnostic Instance Detection for Robotic Manipulation

This study proposed a template-based detection and segmentation approach, which incorporated a multi-level correlation model and a similarity-refine module, for handling the category-agnostic instance.

Go Beyond Point Pairs: A General and Accurate Sim2Real Object Pose Voting Method with Efficient Online Synthetic Training

This paper proposes a novel method for sim-to-real pose estimation, which is effective on both instance-level and category-level settings, based on the point-pair voting scheme from CPPF to vote for object centers, orientations, and scales.

Disentangled Implicit Shape and Pose Learning for Scalable 6D Pose Estimation

This work achieves scalability by disentangling the latent representation of auto-encoding into shape and pose sub-spaces, and shows state-of-the-art performance on two benchmarks containing textureless CAD objects without category and daily objects with categories respectively.

References

SHOWING 1-10 OF 46 REFERENCES

3D Object Detection and Pose Estimation of Unseen Objects in Color Images with Local Surface Embeddings

This work presents an approach for detecting and estimating the 3D poses of objects in images that requires only an untextured CAD model and no training phase for new objects, and relies on an embedding of local 3D geometry to match the CAD models to the input images.

Learning 6D Object Pose Estimation Using 3D Object Coordinates

This work addresses the problem of estimating the 6D Pose of specific objects from a single RGB-D image by presenting a learned, intermediate representation in form of a dense 3D object coordinate labelling paired with a dense class labelling.

BB8: A Scalable, Accurate, Robust to Partial Occlusion Method for Predicting the 3D Poses of Challenging Objects without Using Depth

  • Mahdi RadV. Lepetit
  • Computer Science
    2017 IEEE International Conference on Computer Vision (ICCV)
  • 2017
A novel method for 3D object detection and pose estimation from color images only that uses segmentation to detect the objects of interest in 2D even in presence of partial occlusions and cluttered background and is the first to report results on the Occlusion dataset using color imagesonly.

Segmentation-Driven 6D Object Pose Estimation

This paper introduces a segmentation-driven 6D pose estimation framework where each visible part of the objects contributes a local pose prediction in the form of 2D keypoint locations and uses a predicted measure of confidence to combine these pose candidates into a robust set of 3D-to-2D correspondences.

Making Deep Heatmaps Robust to Partial Occlusions for 3D Object Pose Estimation

A novel method for robust and accurate 3D object pose estimation from a single color image under large occlusions by predicting heatmaps from multiple small patches independently and then computing the 3D pose from these correspondences using a geometric method.

Pose from Shape: Deep Pose Estimation for Arbitrary 3D Objects

This work proposes a completely generic deep pose estimation approach, which does not require the network to have been trained on relevant categories, nor objects in a category to have a canonical pose, and demonstrates that this method boosts performances for supervised category pose estimation on standard benchmarks.

Neural Object Learning for 6D Pose Estimation Using a Few Cluttered Images

This paper proposes a method, Neural Object Learning (NOL), that creates synthetic images of objects in arbitrary poses by combining only a few observations from cluttered images, and a novel refinement step is proposed to align inaccurate poses in source images, which results in better quality images.

Augmented Autoencoders: Implicit 3D Orientation Learning for 6D Object Detection

This novel 3D orientation estimation is based on a variant of the Denoising Autoencoder that is trained on simulated views of a 3D model using Domain Randomization and achieves state-of-the-art performance on the T-LESS dataset both in the RGB and RGB-D domain.

CosyPose: Consistent multi-view multi-object 6D pose estimation

The proposed method, dubbed CosyPose, outperforms current state-of-the-art results for single-view and multi-view 6D object pose estimation by a large margin on two challenging benchmarks: the YCB-Video and T-LESS datasets.

Pose Guided RGBD Feature Learning for 3D Object Pose Estimation

Experimental results show that the features that are learnt guided by poses, are significantly more discriminative than the ones learned in the traditional way, outperforming state-of-the-art works.