• Corpus ID: 13936837

Learning Structured Output Representation using Deep Conditional Generative Models

  title={Learning Structured Output Representation using Deep Conditional Generative Models},
  author={Kihyuk Sohn and Honglak Lee and Xinchen Yan},
Supervised deep learning has been successfully applied to many recognition problems. [] Key Method The model is trained efficiently in the framework of stochastic gradient variational Bayes, and allows for fast prediction using stochastic feed-forward inference. In addition, we provide novel strategies to build robust structured prediction algorithms, such as input noise-injection and multi-scale prediction objective at training. In experiments, we demonstrate the effectiveness of our proposed algorithm in…

Figures and Tables from this paper

Structured Output Learning with Conditional Generative Flows

C-Glow is a conditional generative flow for structured output learning that benefits from the ability of flow-based models to compute p(y|x) exactly and efficiently and a samplebased prediction method is developed, which can use this advantage to do efficient and effective inference.

Structured Output Learning with Conditional Generative Flows

Conditional Glow (c-Glow) is proposed, a conditional generative flow for structured output learning that benefits from the ability of flow-based models to compute p(y|x) exactly and efficiently and can use this advantage to do efficient and effective inference.

Learning Output Embeddings in Structured Prediction

This work proposes to jointly learn a finite approximation of the output embedding and the regression function into the new feature space, and proves that the resulting structured predictor is a consistent estimator, and derive an excess risk bound.

Deep Generative Models: Deterministic Prediction with an Application in Inverse Rendering

The more informative the conditions in terms of object pose are, the less diverse the CVAE predictions are, and this paper analyzes the impact of CVAe's condition on the diversity of solutions given by the designed C VAE in 3D shape inverse rendering as a prediction problem.

Learning Generative Models from Incomplete Data

This thesis introduces a deep generative model, the Variational Auto-decoder (VAD), a variant of the stochastic gradient variational Bayes (SGVB) estimator first introduced by Kingma and Welling in 2013 and shows that the VAD framework is more robust to different rates of missing data than previous generative models for incomplete data.

Multimodal Generative Models for Scalable Weakly-Supervised Learning

A multimodal variational autoencoder that uses a product-of-experts inference network and a sub-sampled training paradigm to solve the multi-modal inference problem and shares parameters to efficiently learn under any combination of missing modalities, thereby enabling weakly-supervised learning.

Conditional Generative Modeling via Learning the Latent Space

This work proposes a novel general-purpose framework for conditional generation in multimodal spaces, that uses latent variables to model generalizable learning patterns while minimizing a family of regression cost functions.


A method to condition generation without retraining the model is developed, combining attribute constraints with a universal “realism” constraint, which enforces similarity to the data distribution, and realistic conditional images are generated from an unconditional variational autoencoder.

Discriminative Multimodal Learning via Conditional Priors in Generative Models

A novel conditional multi-modal discriminative model that uses an informative prior distribution and optimizes a likelihood-free objective function that maximizes mutual information between joint representations and missing modalities is introduced.



Deep Generative Stochastic Networks Trainable by Backprop

Theorems that generalize recent work on the probabilistic interpretation of denoising autoencoders are provided and obtain along the way an interesting justification for dependency networks and generalized pseudolikelihood.

Unsupervised learning of hierarchical representations with convolutional deep belief networks

The convolutional deep belief network is presented, a hierarchical generative model that scales to realistic image sizes and is translation-invariant and supports efficient bottom-up and top-down probabilistic inference.

Improved Multimodal Deep Learning with Variation of Information

This paper proposes a novel multimodal representation learning framework that explicitly aims to minimize the variation of information, and applies this framework to restricted Boltzmann machines and introduces learning methods based on contrastive divergence and multi-prediction training.

Auto-Encoding Variational Bayes

A stochastic variational inference and learning algorithm that scales to large datasets and, under some mild differentiability conditions, even works in the intractable case is introduced.

Learning Stochastic Feedforward Neural Networks

A stochastic feedforward network with hidden layers composed of both deterministic and stochastics variables is proposed that achieves superior performance on synthetic and facial expressions datasets compared to conditional Restricted Boltzmann Machines and Mixture Density Networks.

Stochastic Backpropagation and Approximate Inference in Deep Generative Models

We marry ideas from deep neural networks and approximate Bayesian inference to derive a generalised class of deep, directed generative models, endowed with a new algorithm for scalable inference and

Exploring Compositional High Order Pattern Potentials for Structured Output Learning

Quantitative results on challenging highly variable datasets show that the joint learning and image-dependent high order potentials can improve performance, and a quantitative variability measure is defined to aid in studying this.

Extracting and composing robust features with denoising autoencoders

This work introduces and motivate a new training principle for unsupervised learning of a representation based on the idea of making the learned representations robust to partial corruption of the input pattern.

Semi-supervised Learning with Deep Generative Models

It is shown that deep generative models and approximate Bayesian inference exploiting recent advances in variational methods can be used to provide significant improvements, making generative approaches highly competitive for semi-supervised learning.

Generative Adversarial Nets

We propose a new framework for estimating generative models via an adversarial process, in which we simultaneously train two models: a generative model G that captures the data distribution, and a