• Publications
  • Influence
KNN Matting
TLDR
The matting technique, aptly called KNN matting, capitalizes on the nonlocal principle by using K nearest neighbors (KNN) in matching nonlocal neighborhoods, and contributes a simple and fast algorithm giving competitive results with sparse user markups.
MakeItTalk: Speaker-Aware Talking Head Animation
TLDR
A method that generates expressive talking heads from a single facial image with audio as the only input that is able to synthesize photorealistic videos of entire talking heads with full range of motion and also animate artistic paintings, sketches, 2D cartoon characters, Japanese mangas, stylized caricatures in a single unified framework.
Unified Multisensory Perception: Weakly-Supervised Audio-Visual Video Parsing
TLDR
Experimental results show that the challenging audio-visual video parsing can be achieved even with only video-level weak labels, and the proposed framework can effectively leverage unimodal and cross-modal temporal contexts and alleviate modality bias and noisy labels problems.
AirCode: Unobtrusive Physical Tags for Digital Fabrication
TLDR
A tool that automates the design of air pockets for the user to encode information and demonstrates the tagging technique with applications for metadata embedding, robotic grasping, as well as conveying object affordances.
Motion-Aware KNN Laplacian for Video Matting
TLDR
This paper demonstrates how the nonlocal principle benefits video matting via the KNN Laplacian, which comes with a straightforward implementation using motion-aware K nearest neighbors, which is effective in addressing the fundamental problem of spatio-temporally coherent clusters of moving foreground pixels.
Expediting precomputation for reduced deformable simulation
TLDR
This work presents a complete system of precomputation pipeline as a faster alternative to the classic linear and nonlinear modal analysis, and identifies three bottlenecks in the traditional model reduction precomPUTation, namely modal matrix construction, cubature training, and training dataset generation.
LayerCode: optical barcodes for 3D printed shapes
TLDR
It is shown that LayerCode tags can work on complex, nontrivial shapes, on which all previous tagging mechanisms may fail, and an encoding algorithm is introduced that enables the 3D printing layers to carry information without altering the object geometry.
Deep Audio Prior
TLDR
It is demonstrated that a randomly-initialized neural network can be used with carefully designed audio prior to tackle challenging audio problems such as universal blind source separation, interactive audio editing, audio texture synthesis, and audio co-separation.
Interactive Acoustic Transfer Approximation for Modal Sound
TLDR
This work proposes a new method for interactive and continuous editing as well as exploration of modal sound parameters, and develops a compact, low-memory representation of frequency-varying acoustic transfer values at each key point using Prony series.
Crumpling sound synthesis
TLDR
A physically based algorithm that automatically synthesizes crumpling sounds for a given thin shell animation is proposed, demonstrating the utility of the sound synthesis method in producing realistic sounds at practical computation times.
...
...