Encode, Review, and Decode: Reviewer Module for Caption Generation.

@article{Yang2016EncodeRA,
  title={Encode, Review, and Decode: Reviewer Module for Caption Generation.},
  author={Zhilin Yang and Ye Yuan and Yuexin Wu and Ruslan Salakhutdinov and William W. Cohen},
  journal={CoRR},
  year={2016},
  volume={abs/1605.07912}
}
We propose a novel module, the reviewer module, to improve the encoder-decoder learning framework. The reviewer module is generic, and can be plugged into an existing encoder-decoder model. The reviewer module performs a number of review steps with attention mechanism on the encoder hidden states, and outputs a fact vector after each review step; the fact vectors are used as the input of the attention mechanism in the decoder. We show that the conventional encoderdecoders are a special case of… CONTINUE READING
Highly Cited
This paper has 36 citations. REVIEW CITATIONS

From This Paper

Figures, tables, and topics from this paper.

Explore Further: Topics Discussed in This Paper

Citations

Publications citing this paper.
Showing 1-10 of 26 extracted citations

Knowing When to Look: Adaptive Attention via a Visual Sentinel for Image Captioning

2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) • 2017
View 6 Excerpts
Method Support
Highly Influenced

A Dual Prediction Network for Image Captioning

2018 IEEE International Conference on Multimedia and Expo (ICME) • 2018

References

Publications referenced by this paper.
Showing 1-10 of 21 references

Image Captioning with Semantic Attention

2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) • 2016
View 4 Excerpts
Highly Influenced

Show and tell: A neural image caption generator

2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) • 2015
View 3 Excerpts
Highly Influenced

Deep Visual-Semantic Alignments for Generating Image Descriptions

IEEE Transactions on Pattern Analysis and Machine Intelligence • 2015
View 2 Excerpts

End-to-end memory networks

Sainbayar Sukhbaatar, Jason Weston, Rob Fergus
In NIPS, • 2015
View 1 Excerpt