• Corpus ID: 225062527

LoopReg: Self-supervised Learning of Implicit Surface Correspondences, Pose and Shape for 3D Human Mesh Registration

  title={LoopReg: Self-supervised Learning of Implicit Surface Correspondences, Pose and Shape for 3D Human Mesh Registration},
  author={Bharat Lal Bhatnagar and Cristian Sminchisescu and Christian Theobalt and Gerard Pons-Moll},
We address the problem of fitting 3D human models to 3D scans of dressed humans. Classical methods optimize both the data-to-model correspondences and the human model parameters (pose and shape), but are reliable only when initialized close to the solution. Some methods initialize the optimization based on fully supervised correspondence predictors, which is not differentiable end-to-end, and can only process a single scan at a time. Our main contribution is LoopReg, an end-to-end learning… 

Locally Aware Piecewise Transformation Fields for 3D Human Mesh Registration

Novel piecewise trans-formation fields (PTF), a set of functions that learn 3D translation vectors to map any query point in posed space to its correspond position in rest-pose space, facilitating canonicalized occupancy estimation and results in more accurate surface reconstruction with only half of the parameters compared with the state-of-the-art.

Adjoint Rigid Transform Network: Self-supervised Alignment of 3D Shapes

The Adjoint Rigid Transform (ART) Network is proposed, a neural module which can be integrated with existing 3D networks to significantly boost their performance in tasks such as shape reconstruction, non-rigid registration, and latent disentanglement.

Self-supervised 3D Human Mesh Recovery from Noisy Point Clouds

This paper presents a novel self-supervised approach to reconstruct human shape and pose from noisy point cloud data which treats the process of correspondence search as an implicit probabilistic association by updating the posterior probability of the template model given the input.

imGHUM: Implicit Generative Models of 3D Human Shape and Articulated Pose

ImGHUM is presented, the first holistic generative model of 3D human shape and articulated pose, represented as a signed distance function, and has attached spatial semantics making it straightforward to establish correspondences between different shape instances, thus enabling applications that are difficult to tackle using classical implicit representations.

Learned Vertex Descent: A New Direction for 3D Human Model Fitting

An exhaustive evaluation demonstrates that the proposed novel optimization-based paradigm, dubbed LVD, is able to capture the underlying body of clothed people with very different body shapes, achieving a significant improvement compared to state-of-the-art.

Adjoint Rigid Transform Network: Task-conditioned Alignment of 3D Shapes

The Adjoint Rigid Transform (ART) Network is proposed, a neural module which can be integrated with a variety of 3D networks to significantly boost their performance.

Learning Implicit Functions for Dense 3D Shape Correspondence of Generic Objects

  • Feng LiuXiaoming Liu
  • Computer Science
    IEEE Transactions on Pattern Analysis and Machine Intelligence
  • 2023
The objective of this paper is to learn dense 3D shape correspondence for topology-varying generic objects in an unsupervised manner by implementing dense correspondence through an inverse function mapping from the part embedding vector to a corresponded 3D point.

SCANimate: Weakly Supervised Learning of Skinned Clothed Avatar Networks

SCANimate is presented, an end-to-end trainable framework that takes raw 3D scans of a clothed human and turns them into an animatable avatar that is driven by pose parameters and has realistic clothing that moves and deforms naturally.

Parametric Model Estimation for 3D Clothed Humans from Point Clouds

The experimental results demonstrate that the proposed hierarchical regression approach can accurately predict detailed human shapes from partial point clouds and outperform prior works in the recovery accuracy of 3D human models.

SNARF: Differentiable Forward Skinning for Animating Non-Rigid Neural Implicit Shapes

SNARF is introduced, which combines the advantages of linear blend skinning for polygonal meshes with those of neural implicit surfaces by learning a forward deformation field without direct supervision, allowing for generalization to unseen poses.



3D-CODED: 3D Correspondences by Deep Deformation

This work presents a new deep learning approach for matching deformable shapes by introducing Shape Deformation Networks which jointly encode 3D shapes and correspondences, and shows that this method is robust to many types of perturbations, and generalizes to non-human shapes.

LBS Autoencoder: Self-Supervised Fitting of Articulated Meshes to Point Clouds

This work presents LBS-AE; a self-supervised autoencoding algorithm for fitting articulated mesh models to point clouds that achieves performance that is superior to other unsupervised approaches and comparable to methods using supervised examples.

The Correlated Correspondence Algorithm for Unsupervised Registration of Nonrigid Surfaces

An unsupervised algorithm for registering 3D surface scans of an object undergoing significant deformations that can be used for compelling computer graphics tasks such as interpolation between two scans of a non-rigid object and automatic recovery of articulated object models.

FARM: Functional Automatic Registration Method for 3D Human Bodies

A new method for non‐rigid registration of 3D human shapes that makes use of the functional map representation for encoding and inferring shape maps throughout the registration process, with results in line with, or even surpassing, state‐of‐the‐art methods in the respective areas.

FAUST: Dataset and Evaluation for 3D Mesh Registration

A novel mesh registration technique that combines 3D shape and appearance information to produce high-quality alignments is addressed with a new dataset called FAUST that contains 300 scans of 10 people in a wide range of poses together with an evaluation methodology.

Dense Human Body Correspondences Using Convolutional Networks

This work uses a deep convolutional neural network to train a feature descriptor on depth map pixels, but crucially, rather than training the network to solve the shape correspondence problem directly, it trains it to solve a body region classification problem, modified to increase the smoothness of the learned descriptors near region boundaries.

KillingFusion: Non-rigid 3D Reconstruction without Correspondences

A novel regularizer is proposed that imposes local rigidity by requiring the deformation to be a smooth and approximately Killing vector field, i.e. generating nearly isometric motions and enforcing that the level set property of unity gradient magnitude is preserved over iterations.

Efficient and precise interactive hand tracking through joint, continuous optimization of pose and correspondences

Quantitative assessments on standard datasets show that the new approach exceeds the state of the art in accuracy, and runs in real-time on CPU only, which frees up the commonly over-burdened GPU for experience designers.

GHUM & GHUML: Generative 3D Human Shape and Articulated Pose Models

A statistical, articulated 3D human shape modeling pipeline, within a fully trainable, modular, deep learning framework, that supports facial expression analysis, as well as body shape and pose estimation.

Dynamic FAUST: Registering Human Bodies in Motion

This work proposes a new mesh registration method that uses both 3D geometry and texture information to register all scans in a sequence to a common reference topology, and shows how using geometry alone results in significant errors in alignment when the motions are fast and non-rigid.