• Corpus ID: 232240572

PC-HMR: Pose Calibration for 3D Human Mesh Recovery from 2D Images/Videos

@article{Luan2021PCHMRPC,
  title={PC-HMR: Pose Calibration for 3D Human Mesh Recovery from 2D Images/Videos},
  author={Tianyu Luan and Yali Wang and Junhao Zhang and Zhe Wang and Zhipeng Zhou and Yu Qiao},
  journal={ArXiv},
  year={2021},
  volume={abs/2103.09009}
}
The end-to-end Human Mesh Recovery (HMR) approach (Kanazawa et al. 2018) has been successfully used for 3D body reconstruction. However, most HMR-based frameworks reconstruct human body by directly learning mesh parameters from images or videos, while lacking explicit guidance of 3D human pose in visual data. As a result, the generated mesh often exhibits incorrect pose for complex activities. To tackle this problem, we propose to exploit 3D pose to calibrate human mesh. Specifically, we develop… 

Figures and Tables from this paper

MeshMAE: Masked Autoencoders for 3D Mesh Data Analysis
TLDR
Inspired by MAE, how pre-training on 3D mesh data with the Transformer-based structure benefits downstream3D mesh analysis tasks is explored and the method is named MeshMAE, which can yield state-of-the-art or comparable performance onMesh analysis tasks, i.e., classification and segmentation.
Parallel‐branch network for 3D human pose and shape estimation in video
TLDR
A parallel‐branch network including a designed spatial–temporal (ST) branch and a SMPL branch is proposed that performs the 2D‐to‐3D lifting for more accurate joint prediction, by the designed spatial transformer and temporal transformer.
The Best of Both Worlds: Combining Model-based and Nonparametric Approaches for 3D Human Body Estimation
TLDR
This framework leverages the best of non-parametric and model-based methods and is also robust to partial occlusion and outperforms existing 3D human estimation methods on multiple public benchmarks.
Recovering 3D Human Mesh from Monocular Images: A Survey
TLDR
This is the first survey to focus on the task of monocular 3D human mesh recovery and starts with the introduction of body models and then elaborate recovery frameworks and training objectives by providing in-depth analyses of their strengths and weaknesses.
Pose-guided Generative Adversarial Net for Novel View Action Synthesis
TLDR
A novel framework named Pose-guided Action Separable Generative Adversarial Net (PAS-GAN), which utilizes pose to alleviate the difficulty of this task and conducts extensive experiments on two large-scale multi-view human action datasets, NTU-RGBD and PKU-MMD, demonstrating the effectiveness of PAS-gan which outperforms existing approaches.
PyMAF: 3D Human Pose and Shape Regression with Pyramidal Mesh Alignment Feedback Loop
TLDR
A Pyramidal Mesh Alignment Feedback (PyMAF) loop to leverage a feature pyramid and rectify the predicted parameters explicitly based on the mesh-image alignment status in the authors' deep regressor is proposed.

References

SHOWING 1-10 OF 46 REFERENCES
Learning 3D Human Dynamics From Video
TLDR
The approach is designed so it can learn from videos with 2D pose annotations in a semi-supervised manner and obtain state-of-the-art performance on the 3D prediction task without any fine-tuning.
3D Human Pose Estimation in Video With Temporal Convolutions and Semi-Supervised Training
In this work, we demonstrate that 3D poses in video can be effectively estimated with a fully convolutional model based on dilated temporal convolutions over 2D keypoints. We also introduce
Cascaded Pyramid Network for 3D Human Pose Estimation Challenge
TLDR
A top-bottom based two-stage 3D estimation framework that achieves promising results with mean per joint position error at 42.39 on the validation dataset on `3D Human Pose Estimation within the ECCV 2018 PoseTrack Challenge.
End-to-End Recovery of Human Shape and Pose
TLDR
This work introduces an adversary trained to tell whether human body shape and pose parameters are real or not using a large database of 3D human meshes, and produces a richer and more useful mesh representation that is parameterized by shape and 3D joint angles.
SMPL: a skinned multi-person linear model
TLDR
The Skinned Multi-Person Linear model (SMPL) is a skinned vertex-based model that accurately represents a wide variety of body shapes in natural human poses that is compatible with existing graphics pipelines and iscompatible with existing rendering engines.
Learning to Reconstruct 3D Human Pose and Shape via Model-Fitting in the Loop
TLDR
The core of the proposed approach SPIN (SMPL oPtimization IN the loop) is that the two paradigms can form a strong collaboration, and better network estimates can lead the optimization to better solutions, while more accurate optimization fits provide better supervision for the network.
Human Mesh Recovery from Monocular Images via a Skeleton-disentangled Representation
TLDR
A skeleton-disentangled based framework is proposed, which divides this task into multi-level spatial and temporal granularity in a decoupling manner, and an effective and pluggable "disentangling the skeleton from the details" (DSD) module is proposed.
A Simple Yet Effective Baseline for 3d Human Pose Estimation
TLDR
The results indicate that a large portion of the error of modern deep 3d pose estimation systems stems from their visual analysis, and suggests directions to further advance the state of the art in 3d human pose estimation.
Learning from Synthetic Humans
TLDR
This work presents SURREAL (Synthetic hUmans foR REAL tasks): a new large-scale dataset with synthetically-generated but realistic images of people rendered from 3D sequences of human motion capture data and shows that CNNs trained on this synthetic dataset allow for accurate human depth estimation and human part segmentation in real RGB images.
Deep HighResolution Representation Learning for Human Pose Estimation
  • CVPR.
  • 2019
...
...