Per Garment Capture and Synthesis for Real-time Virtual Try-on

  title={Per Garment Capture and Synthesis for Real-time Virtual Try-on},
  author={Toby Long Hin Chong and I-Chao Shen and Nobuyuki Umetani and Takeo Igarashi},
  journal={The 34th Annual ACM Symposium on User Interface Software and Technology},
  • T. Chong, I-Chao Shen, T. Igarashi
  • Published 10 September 2021
  • Computer Science
  • The 34th Annual ACM Symposium on User Interface Software and Technology
Virtual try-on is a promising application of computer graphics and human computer interaction that can have a profound real-world impact especially during this pandemic. Existing image-based works try to synthesize a try-on image from a single image of a target garment, but it inherently limits the ability to react to possible interactions. It is difficult to reproduce the change of wrinkles caused by pose and body size change, as well as pulling and stretching of the garment by hand. In this… 
1 Citations
Real-time Image-based Virtual Try-on with Measurement Garment
This work presents a real-time image-based virtual try-on system composed of two parts, i.e., photo-realistic clothed person image synthesis for the customers to experience the try- on result and garment capturing for the retailers to capture the rich deformations of the target garment.


Virtual Try-On through Image-Based Rendering
This work transfers the appearance of a garment recorded from one user to another by matching input and recorded frames, image-based visual hull rendering, and online registration methods, which is suitable for a wide range of clothes and complex appearances, allows arbitrary viewing angles, and requires only little manual input.
Physics-Inspired Garment Recovery from a Single-View Image
This work proposes a method that is able to compute a 3D model of a human body and its outfit from a single photograph with little human interaction and demonstrates the effectiveness of the algorithm by re-purposing the reconstructed garments for virtual try-on and garment transfer applications and for cloth animation on digital characters.
ShineOn: Illuminating Design Choices for Practical Video-based Virtual Clothing Try-on
This work investigates the effect of different pose annotations, self-attention layer placement, and activation functions on the quantitative and qualitative performance of video virtual try-on, and finds that Dense-Pose annotations not only enhance face details but also decrease memory usage and training time.
Garment modeling with a depth camera
This work studies the modeling of real garments and develops a system that is intuitive to use even for novice users, including garment component detectors and design attribute classifiers learned from a manually labeled garment image database.
3D Virtual Garment Modeling from RGB Images
A novel and efficient approach that automatically constructs 3D models for garments using only RGB images and proposes a multi-task learning network called JFNet that jointly identifies the landmarks of the garment as well as parses the garment into semantic part segments.
Toward Characteristic-Preserving Image-based Virtual Try-On Network
A new fully-learnable Characteristic-Preserving Virtual Try-On Network (CP-VTON) for addressing all real-world challenges in this task, and achieves the state-of-the-art virtual try-on performance both qualitatively and quantitatively.
Deep Fashion3D: A Dataset and Benchmark for 3D Garment Reconstruction from Single Images
Deep Fashion3D is introduced, the largest collection to date of 3D garment models, with the goal of establishing a novel benchmark and dataset for the evaluation of image-based garment reconstruction systems.
Towards Photo-Realistic Virtual Try-On by Adaptively Generating↔Preserving Image Content
This work proposes a novel visual try-on network, namely Adaptive Content Generating and Preserving Network (ACGPN), which can generate photo-realistic images with much better perceptual quality and richer fine-details.
Image-Based Virtual Fitting System with Garment Image Reshaping
An image-based virtual fitting system to reproduce the appearance of fitting during online shopping for garments by reshaping the garment image based on human body shapes of the fashion model and the customer to make the fitting result more realistic.
Towards Multi-Pose Guided Virtual Try-On Network
This paper makes the first attempt towards a multi-pose guided virtual try-on system, which enables clothes to transfer onto a person with diverse poses, and significantly outperforms all state-of-the-art methods both qualitatively and quantitatively.