Learning skeletal articulations with neural blend shapes

@article{Li2021LearningSA,
  title={Learning skeletal articulations with neural blend shapes},
  author={Peizhuo Li and Kfir Aberman and Rana Hanocka and Libin Liu and Olga Sorkine-Hornung and Baoquan Chen},
  journal={ACM Transactions on Graphics (TOG)},
  year={2021},
  volume={40},
  pages={1 - 15}
}
Animating a newly designed character using motion capture (mocap) data is a long standing problem in computer animation. A key consideration is the skeletal structure that should correspond to the available mocap data, and the shape deformation in the joint regions, which often requires a tailored, pose-specific refinement. In this work, we develop a neural technique for articulating 3D characters using enveloping with a pre-defined skeletal structure which produces high quality pose dependent… 

Skeleton-free Pose Transfer for Stylized 3D Characters

This work presents the first method that automatically transfers poses between stylized 3D characters without skeletal rigging, and proposes a novel pose transfer network that predicts the character skinning weights and deformation transformations jointly to articulate the target character to match the desired pose.

3D shape deformation using stick figures

WOC: A Handy Webcam-based 3D Online Chatroom

We develop WOC, a webcam-based 3D virtual online chatroom for multi-person interaction, which captures the 3D motion of users and drives their individual 3D virtual avatars in real-time. Compared to

Neural jacobian fields

This paper introduces a framework designed to accurately predict piecewise linear mappings of arbitrary meshes via a neural network, enabling training and evaluating over heterogeneous collections of

The Analysis of Two-Way E-Commerce Credit Evaluation Model Based on the C2C Mode

As an important mode of e-commerce, C2C has become a trading mechanism favored by consumers. However, for C2C transaction in a virtual environment, there is an issue of congenital transaction

Character articulation through profile curves

A new approach for character articulation that produces detail-preserving deformations fully controlled by 3D curves that profile the deforming surface and formulate a deformation optimization that reconstructs surface details while conforming to the rigged curvenets.

An overview of machine learning and other data-based methods for spatial audio capture, processing, and reproduction

This article reviews the most important application domains of data-based spatial audio including well-established methods that employ conventional signal processing while paying special attention to the most recent achievements that make use of machine learning.

Cleanup Sketched Drawings: Deep Learning-Based Model

The efficiency and effectiveness of using deep learning in cleaning and improving the roughly drawn image in an automatic way is demonstrated, and an enhanced FCNN model reported better accuracy is observed.

References

SHOWING 1-10 OF 61 REFERENCES

RigNet: Neural Rigging for Articulated Characters

RigNet, an end-to-end automated method for producing animation rigs from input character models, predicts a skeleton that matches the animator expectations in joint placement and topology and estimates surface skin weights based on the predicted skeleton.

SMPL: a skinned multi-person linear model

The Skinned Multi-Person Linear model (SMPL) is a skinned vertex-based model that accurately represents a wide variety of body shapes in natural human poses that is compatible with existing graphics pipelines and iscompatible with existing rendering engines.

Joint-dependent local deformations for hand animation and object grasping

Algorithms and methods are presented for animating the hands of a synthetic actor. The algorithms allow the land to move and grasp objects; they also compute deformations of the hand as it moves. The

STAR: Sparse Trained Articulated Human Body Regressor

This work defines per-joint pose correctives and learns the subset of mesh vertices that are influenced by each joint movement that results in more realistic deformations and significantly reduces the number of model parameters to 20% of SMPL.

Skinning with dual quaternions

This paper presents a novel GPU-friendly skinning algorithm based on dual quaternions and shows that this approach solves the artifacts of linear blend skinning at minimal additional cost.

Pose space deformation: a unified approach to shape interpolation and skeleton-driven deformation

Pose space deformation generalizes and improves upon both shape interpolation and common skeleton-driven deformation techniques and achieves improved expressive power and direct manipulation of the desired shapes yet the performance associated with traditionalshape interpolation is achievable.

Context‐Aware Skeletal Shape Deformation

We describe a system for the animation of a skeleton‐controlled articulated object that preserves the fine geometric details of the object skin and conforms to the characteristic shapes of the object

Complementary dynamics

A novel approach to enrich arbitrary rig animations with elastodynamic secondary effects that plugs into non-linear FEM simulations, geometric as-rigid-as-possible energies, or mass-spring models and ensures that the additional dynamic motions do not undo the rig animation.

Fast and deep facial deformations

This paper presents a method using convolutional neural networks for approximating the mesh deformations of characters' faces that runs up to 17 times faster than the original facial rig while still maintaining a high level of fidelity to the original rig.

Accurate face rig approximation with deep differential subspace reconstruction

A rig approximation method that addresses issues by learning localized shape information in differential coordinates and, separately, a subspace for mesh reconstruction that can reconstruct both face and body deformations with high fidelity and does not require a set of well-posed animation examples.
...