Diverse and Accurate Image Description Using a Variational Auto-Encoder with an Additive Gaussian Encoding Space

@inproceedings{Wang2017DiverseAA,
  title={Diverse and Accurate Image Description Using a Variational Auto-Encoder with an Additive Gaussian Encoding Space},
  author={Liwei Wang and Alexander G. Schwing and Svetlana Lazebnik},
  booktitle={NIPS},
  year={2017}
}
This paper explores image caption generation using conditional variational autoencoders (CVAEs). Standard CVAEs with a fixed Gaussian prior yield descriptions with too little variability. Instead, we propose two models that explicitly structure the latent space around K components corresponding to different types of image content, and combine components to create priors for images that contain multiple types of content simultaneously (e.g., several kinds of objects). Our first model uses a… CONTINUE READING
Highly Cited
This paper has 24 citations. REVIEW CITATIONS
Related Discussions
This paper has been referenced on Twitter 9 times. VIEW TWEETS

Citations

Publications citing this paper.
Showing 1-10 of 18 extracted citations

From image to language and back again

Natural Language Engineering • 2018
View 2 Excerpts
Highly Influenced

Convolutional Image Captioning

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition • 2018
View 1 Excerpt

Discriminability Objective for Training Descriptive Captions

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition • 2018
View 1 Excerpt
Method Support

References

Publications referenced by this paper.
Showing 1-10 of 37 references

Creativity: Generating Diverse Questions Using Variational Autoencoders

2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) • 2017
View 6 Excerpts
Highly Influenced

Towards Diverse and Natural Image Descriptions via a Conditional GAN

2017 IEEE International Conference on Computer Vision (ICCV) • 2017
View 5 Excerpts
Highly Influenced

Improved Image Captioning via Policy Gradient optimization of SPIDEr

2017 IEEE International Conference on Computer Vision (ICCV) • 2017
View 1 Excerpt

Learning Diverse Image Colorization

2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) • 2017
View 1 Excerpt

Show and Tell: Lessons Learned from the 2015 MSCOCO Image Captioning Challenge

IEEE Transactions on Pattern Analysis and Machine Intelligence • 2017
View 1 Excerpt