UltraPose: Synthesizing Dense Pose with 1 Billion Points by Human-body Decoupling 3D Model

  title={UltraPose: Synthesizing Dense Pose with 1 Billion Points by Human-body Decoupling 3D Model},
  author={Haonan Yan and Jiaqi Chen and Xujie Zhang and Shengkai Zhang and Nianhong Jiao and Xiaodan Liang and Tianxiang Zheng},
  journal={2021 IEEE/CVF International Conference on Computer Vision (ICCV)},
  • Haonan YanJiaqi Chen Tianxiang Zheng
  • Published 1 October 2021
  • Computer Science
  • 2021 IEEE/CVF International Conference on Computer Vision (ICCV)
Recovering dense human poses from images plays a critical role in establishing an image-to-surface correspondence between RGB images and the 3D surface of the human body, serving the foundation of rich real-world applications, such as virtual humans, monocular-to-3d reconstruction. However, the popular DensePose-COCO dataset relies on a sophisticated manual annotation system, leading to severe limitations in acquiring the denser and more accurate annotated pose resources. In this work, we… 

Figures and Tables from this paper

BodyMap: Learning Full-Body Dense Correspondence Map

A novel network architecture with Vision Transformers that learn fine-level features on a continuous body surface and outperforms prior work on various metrics and datasets, including DensePose-COCO by a large margin.

ArtiBoost: Boosting Articulated 3D Hand-Object Pose Estimation via Online Exploration and Synthesis

  • Kailin LiLixin Yang Cewu Lu
  • Computer Science
    2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2022
This work proposes ArtiBoost, a lightweight online data enhancement method that performs data exploration and synthesis within a learning pipeline, and those synthetic data are blended into real-world source data for training.

HSPACE: Synthetic Parametric Humans Animated in Complex Environments

A large-scale analysis of the impact of synthetic data, in connection with real data and weak supervision, underlines the considerable potential for continuing quality improvements and limiting the sim-to-real gap, in this practical setting, inconnection with increased model capacity.

TemporalUV: Capturing Loose Clothing with Temporally Coherent UV Coordinates

A novel approach to generate temporally coherent UV coordinates for loose clothing by implementing a differentiable pipeline to learn UV mapping between a sequence of RGB inputs and textures via UV coordinates.

Human Action Recognition of Triangle Mesh Sequence Representation

: Considering the existing research works of human motion recognition are based on skeleton and video representations, a human motion classification method for triangle mesh sequence representation



DensePose: Dense Human Pose Estimation in the Wild

This work establishes dense correspondences between an RGB image and a surface-based representation of the human body, a task referred to as dense human pose estimation, and improves accuracy through cascading, obtaining a system that delivers highly-accurate results at multiple frames per second on a single gpu.

HoloPose: Holistic 3D Human Reconstruction In-The-Wild

A part-based model for 3D model parameter regression that allows the HoloPose method to operate in-the-wild, gracefully handling severe occlusions and large pose variation is introduced and validated on challenging benchmarks.

Dense Pose Transfer

This work proposes a combination of surface-based pose estimation and deep generative models that allows us to perform accurate pose transfer, i.e. synthesize a new image of a person based on a single image of that person and theimage of a pose donor.

Slim DensePose: Thrifty Learning From Sparse Annotations and Motion Cues

It is demonstrated that if annotations are collected in video frames, their efficacy can be multiplied for free by using motion cues, and that motion cues help much more when they are extracted from videos.

Learning from Synthetic Humans

This work presents SURREAL (Synthetic hUmans foR REAL tasks): a new large-scale dataset with synthetically-generated but realistic images of people rendered from 3D sequences of human motion capture data and shows that CNNs trained on this synthetic dataset allow for accurate human depth estimation and human part segmentation in real RGB images.

TexturePose: Supervising Human Mesh Estimation With Texture Consistency

This work proposes a natural form of supervision, that capitalizes on the appearance constancy of a person among different frames (or viewpoints) and achieves state-of-the-art results among model-based pose estimation approaches in different benchmarks.

Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments

We introduce a new dataset, Human3.6M, of 3.6 Million accurate 3D Human poses, acquired by recording the performance of 5 female and 6 male subjects, under 4 different viewpoints, for training

DenseReg: Fully Convolutional Dense Shape Regression In-the-Wild

The proposed system, called DenseReg, allows us to estimate dense image-to-template correspondences in a fully convolutional manner and can provide useful correspondence information as a stand-alone system, while when used as an initialization for Statistical Deformable Models the authors obtain landmark localization results that largely outperform the current state-of-the-art on the challenging 300W benchmark.

Coordinate-Based Texture Inpainting for Pose-Guided Human Image Generation

A new deep learning approach to pose-guided resynthesis of human photographs using a fully-convolutional architecture with deformable skip connections guided by the estimated correspondence field and a new inpainting method that completes the texture of the human body.

Continuous Surface Embeddings

This work proposes a new, learnable image-based representation of dense correspondences and demonstrates that the proposed approach performs on par or better than the state-of-the-art methods for dense pose estimation for humans, while being conceptually simpler.