End-to-End Hand Mesh Recovery From a Monocular RGB Image

@article{Zhang2019EndtoEndHM,
  title={End-to-End Hand Mesh Recovery From a Monocular RGB Image},
  author={Xiong Zhang and Qiang Li and Wenbo Zhang and Wen Zheng},
  journal={2019 IEEE/CVF International Conference on Computer Vision (ICCV)},
  year={2019},
  pages={2354-2364}
}
  • Xiong Zhang, Qiang Li, +1 author W. Zheng
  • Published 25 February 2019
  • Computer Science
  • 2019 IEEE/CVF International Conference on Computer Vision (ICCV)
In this paper, we present a HAnd Mesh Recovery (HAMR) framework to tackle the problem of reconstructing the full 3D mesh of a human hand from a single RGB image. [...] Key Method By utilizing this mesh representation, we can easily compute the 3D joint locations via linear interpolations between the vertexes of the mesh, while obtain the 2D joint locations with a projection of the 3D this http URL this end, a differentiable re-projection loss can be defined in terms of the derived representations and the ground…Expand
3D hand mesh reconstruction from a monocular RGB image
TLDR
This work proposes a novel method to reconstruct a 3D hand mesh from a single monocular RGB image and uses the CycleGAN network to transform the data domain of real-world images to that of the synthesis dataset. Expand
Im2Mesh GAN: Accurate 3D Hand Mesh Recovery from a Single RGB Image
TLDR
This work proposes a new type of GAN called Im2Mesh GAN to learn the mesh through end-to-end adversarial training, which is able to capture the topological relationship among the mesh vertices by interpreting the mesh as a graph. Expand
Model-based 3D Hand Reconstruction via Self-Supervised Learning
TLDR
This work proposes S2HAND, a self-supervised 3D hand reconstruction network that can jointly estimate pose, shape, texture, and the camera viewpoint and utilizes the consistency between 2D and 3D representations and a set of novel losses to rationalize outputs of the neural network. Expand
Towards Accurate Alignment in Real-time 3D Hand-Mesh Reconstruction
  • Xiao Tang, Tianyu Wang, Chi-Wing Fu
  • Computer Science
  • ArXiv
  • 2021
TLDR
The quality of the results outperforms the state-of-the-art methods on handmesh/pose precision and hand-image alignment and several real-time AR scenarios are showcased. Expand
HandVoxNet: Deep Voxel-Based Network for 3D Hand Shape and Pose Estimation From a Single Depth Map
TLDR
This work proposes a novel architecture with 3D convolutions trained in a weakly-supervised manner that produces visually more reasonable and realistic hand shapes on NYU and BigHand2.2M datasets compared to the existing approaches. Expand
Realistic Hands: A Hybrid Model for 3D Hand Reconstruction
TLDR
A hybrid approach is proposed, utilizing a deep neural network and differential rendering based optimization to demonstrably achieve the best of both worlds in estimating 3D hand meshes from RGB images robustly and exploring Virtual Reality (VR) as an application. Expand
HTML: A Parametric Hand Texture Model for 3D Hand Reconstruction and Personalization
3D hand reconstruction from images is a widely-studied problem in computer vision and graphics, and has a particularly high relevance for virtual and augmented reality. Although several 3D handExpand
MVHM: A Large-Scale Multi-View Hand Mesh Benchmark for Accurate 3D Hand Pose Estimation
TLDR
A spin match algorithm is designed that enables a rigid mesh model matching with any target mesh ground truth and an efficient pipeline is proposed to generate a large-scale multi-view hand mesh (MVHM) dataset with accurate 3D hand mesh and joint labels. Expand
I2UV-HandNet: Image-to-UV Prediction Network for Accurate and High-fidelity 3D Hand Mesh Modeling
  • Ping Chen, Yujin Chen, +4 authors Yong Tan
  • Computer Science
  • 2021
TLDR
This study proposes an I2UV-HandNet model for accurate hand pose and shape estimation as well as 3D hand super-resolution reconstruction and presents the first UV-based 3DHandNet representation, which is demonstrated to have characterization capability for the first time. Expand
HandVoxNet++: 3D Hand Shape and Pose Estimation using Voxel-Based Neural Networks
TLDR
This paper develops HandVoxNet++, a voxel-based deep network with 3D and graph convolutions trained in a fully supervised manner, and gains 41:09% and 13:7% higher shape alignment accuracy on SynHand5M and HANDS19 datasets, respectively. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 71 REFERENCES
End-to-End Recovery of Human Shape and Pose
TLDR
This work introduces an adversary trained to tell whether human body shape and pose parameters are real or not using a large database of 3D human meshes, and produces a richer and more useful mesh representation that is parameterized by shape and 3D joint angles. Expand
3D Hand Shape and Pose Estimation From a Single RGB Image
TLDR
This work proposes a Graph Convolutional Neural Network (Graph CNN) based method to reconstruct a full 3D mesh of hand surface that contains richer information of both 3D hand shape and pose and proposes a weakly-supervised approach by leveraging the depth map as a weak supervision in training. Expand
3D Hand Shape and Pose From Images in the Wild
TLDR
This work presents the first end-to-end deep learning based method that predicts both 3D hand shape and pose from RGB images in the wild, consisting of the concatenation of a deep convolutional encoder, and a fixed model-based decoder. Expand
Using a Single RGB Frame for Real Time 3D Hand Pose Estimation in the Wild
TLDR
This work capitalize on the latest advancements of deep learning, combining them with the power of generative hand pose estimation techniques to achieve real-time monocular 3D hand Pose estimation in unrestricted scenarios. Expand
GANerated Hands for Real-Time 3D Hand Tracking from Monocular RGB
TLDR
This work proposes a novel approach for the synthetic generation of training data that is based on a geometrically consistent image-to-image translation network, and uses a neural network that translates synthetic images to "real" images, such that the so-generated images follow the same statistical distribution as real-world hand images. Expand
Learning to Estimate 3D Hand Pose from Single RGB Images
TLDR
A deep network is proposed that learns a network-implicit 3D articulation prior that yields good estimates of the 3D pose from regular RGB images, and a large scale 3D hand pose dataset based on synthetic hand models is introduced. Expand
Learning to Estimate 3D Human Pose and Shape from a Single Color Image
TLDR
This work addresses the problem of estimating the full body 3D human pose and shape from a single color image and proposes an efficient and effective direct prediction method based on ConvNets, incorporating a parametric statistical body shape model (SMPL) within an end-to-end framework. Expand
Weakly-Supervised 3D Hand Pose Estimation from Monocular RGB Images
TLDR
A weakly-supervised method, adaptating from fully-annotated synthetic dataset toWeakly-labeled real-world dataset with the aid of a depth regularizer, which generates depth maps from predicted 3D pose and serves as weak supervision for3D pose regression. Expand
Cross-modal Deep Variational Hand Pose Estimation
The human hand moves in complex and highdimensional ways, making estimation of 3D hand pose configurations from images alone a challenging task. In this work we propose a method to learn aExpand
Self-Supervised Multi-level Face Model Learning for Monocular Reconstruction at Over 250 Hz
TLDR
This first approach that jointly learns a regressor for face shape, expression, reflectance and illumination on the basis of a concurrently learned parametric face model is presented, which compares favorably to the state-of-the-art in terms of reconstruction quality, better generalizes to real world faces, and runs at over 250 Hz. Expand
...
1
2
3
4
5
...