Visually-Aware Fashion Recommendation and Design with Generative Image Models

  title={Visually-Aware Fashion Recommendation and Design with Generative Image Models},
  author={Wang-Cheng Kang and Chen Fang and Zhaowen Wang and Julian McAuley},
  journal={2017 IEEE International Conference on Data Mining (ICDM)},
Building effective recommender systems for domains like fashion is challenging due to the high level of subjectivity and the semantic complexity of the features involved (i.e., fashion styles). Recent work has shown that approaches to 'visual' recommendation (e.g. clothing, art, etc.) can be made more accurate by incorporating visual signals directly into the recommendation objective, using 'off-the-shelf' feature representations derived from deep networks. Here, we seek to extend this… 

Figures and Tables from this paper

CuratorNet: Visually-aware Recommendation of Art Images

CuratorNet is a neural network architecture for visually-aware recommendation of art images that achieves the best performance among several baselines, including the state-of-the-art model VBPR.

Personalized Fashion Recommendation with Visual Explanations based on Multimodal Attention Network: Towards Visually Explainable Recommendation

This paper proposes a novel neural architecture for fashion recommendation based on both image region-level features and user review information that can not only provide accurate recommendation, but also can accompany each recommended item with novel visual explanations.

End-to-End Image-Based Fashion Recommendation

A simple yet effective attribute-aware model that incorporates image features for better item representation learning in item recommendation tasks is proposed and significantly outperforms all state-of-the-art image-based models.

Improving Outfit Recommendation with Co-supervision of Fashion Generation

A neural co-supervision learning framework, called the FAshion Recommendation Machine (FARM), which improves visual understanding by incorporating the supervision of generation loss, and enhances visual matching by introducing a novel layer-to-layer matching mechanism to fuse aesthetic information more effectively.

Complete the Look: Scene-Based Complementary Product Recommendation

This work designs an approach to extract training data for a new task called ‘Complete the Look’, which seeks to recommend visually compatible products based on scene images, and proposes a novel way to learn the scene-product compatibility from fashion or interior design images.

A Study on the Relative Importance of Convolutional Neural Networks in Visually-Aware Recommender Systems

This work evaluates the recommendation performance of a suite of VRSs using several pretrained CNNs as the image feature extractors on two datasets from a real-world e-commerce platform and proposes a novel qualitative and quantitative evaluation paradigm to assess the visual diversity of recommended items compared to the interacted user’s items.

Personalized Fashion Design

This work proposes a personalized fashion design framework with the help of generative adversarial training and proposes a matching network which simultaneously models the compatibility between fashion items and learns users' preference representations.

Visually-aware Recommendation with Aesthetic Features

This work addresses the need of modeling aesthetic information in visually-aware recommender systems by incorporating the aesthetic features extracted from product images by a deep aesthetic network into recommender system and designing a new tensor decomposition model for implicit feedback data.

UvA-DARE (Digital Academic Repository) Explainable Fashion Recommendation with Joint Outfit Matching and Comment Generation

A novel neural network framework, neural fashion recommendation (NFR), that simultaneously provides fashion recommendations and generates abstractive comments that achieves impressive ROUGE and BLEU scores in comparison to human-written comments is proposed.

Deep Fashion Recommendation System with Style Feature Decomposition

This work proposes a style feature extraction (SFE) layer, which effectively decomposes the clothes vector into style and category, and extracts and removes the category information from the clothes vectors to obtain more accurate style information.



Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filtering

This paper builds novel models for the One-Class Collaborative Filtering setting, where the goal is to estimate users' fashion-aware personalized ranking functions based on their past feedback and combines high-level visual features extracted from a deep convolutional neural network, users' past feedback, as well as evolving trends within the community.

Vista: A Visually, Socially, and Temporally-aware Model for Artistic Recommendation

This paper builds large-scale recommender systems to model the dynamics of a vibrant digital art community, Behance, consisting of tens of millions of interactions (clicks and 'appreciates') of users toward digital art.

Learning Visual Clothing Style with Heterogeneous Dyadic Co-Occurrences

With the rapid proliferation of smart mobile devices, users now take millions of photos every day. These include large numbers of clothing and accessory images. We would like to answer questions like

VBPR: Visual Bayesian Personalized Ranking from Implicit Feedback

This paper proposes a scalable factorization model to incorporate visual signals into predictors of people's opinions, which is applied to a selection of large, real-world datasets and makes use of visual features extracted from product images using (pre-trained) deep networks.

Comparative Deep Learning of Hybrid Representations for Image Recommendations

A dual-net deep network is designed, in which the two sub-networks map input images and preferences of users into a same latent semantic space, and then the distances between images and users in the latent space are calculated to make decisions.

Image-Based Recommendations on Styles and Substitutes

The approach is not based on fine-grained modeling of user annotations but rather on capturing the largest dataset possible and developing a scalable method for uncovering human notions of the visual relationships within.

Learning Fashion Compatibility with Bidirectional LSTMs

This paper proposes to jointly learn a visual-semantic embedding and the compatibility relationships among fashion items in an end-to-end fashion and trains a bidirectional LSTM (Bi-LSTM) model to sequentially predict the next item conditioned on previous ones to learn their compatibility relationships.

Mining Fashion Outfit Composition Using an End-to-End Deep Learning Approach on Set Data

A machine learning system to compose fashion outfits automatically to score fashion outfit candidates based on the appearances and metadata and achieves an AUC of 85% for the scoring component, and an accuracy of 77% for a constrained composition task.

What Your Images Reveal: Exploiting Visual Contents for Point-of-Interest Recommendation

A new framework Visual Content Enhanced POI recommendation (VPOI), which incorporates visual contents for POI recommendations, is proposed, which demonstrates the effectiveness of the proposed framework on real-world datasets.

Return of the Devil in the Details: Delving Deep into Convolutional Nets

It is shown that the data augmentation techniques commonly applied to CNN-based methods can also be applied to shallow methods, and result in an analogous performance boost, and it is identified that the dimensionality of the CNN output layer can be reduced significantly without having an adverse effect on performance.