Hierarchically Structured Reinforcement Learning for Topically Coherent Visual Story Generation

@inproceedings{Huang2019HierarchicallySR,
  title={Hierarchically Structured Reinforcement Learning for Topically Coherent Visual Story Generation},
  author={Qiuyuan Huang and Zhe Gan and Asli Çelikyilmaz and Dapeng Wu and Jianfeng Wang and Xiaodong He},
  booktitle={AAAI},
  year={2019}
}
  • Qiuyuan Huang, Zhe Gan, +3 authors Xiaodong He
  • Published in AAAI 2019
  • Computer Science
  • We propose a hierarchically structured reinforcement learning approach to address the challenges of planning for generating coherent multi-sentence stories for the visual storytelling task. [...] Key Method The high-level decoder constructs a plan by generating a semantic concept (i.e., topic) for each image in sequence. The low-level decoder generates a sentence for each image using a semantic compositional network, which effectively grounds the sentence generation conditioned on the topic.Expand Abstract

    Figures, Tables, and Topics from this paper.

    Citations

    Publications citing this paper.
    SHOWING 1-10 OF 26 CITATIONS, ESTIMATED 61% COVERAGE

    Emotion Reinforced Visual Storytelling

    VIEW 2 EXCERPTS
    CITES BACKGROUND

    StoryGAN: A Sequential Conditional GAN for Story Visualization

    VIEW 1 EXCERPT
    CITES BACKGROUND

    What Makes A Good Story? Designing Composite Rewards for Visual Storytelling

    VIEW 9 EXCERPTS
    CITES METHODS & BACKGROUND

    Incorporating Textual Evidence in Visual Storytelling

    VIEW 3 EXCERPTS
    CITES BACKGROUND & METHODS

    Visual Storytelling via Predicting Anchor Word Embeddings in the Stories

    VIEW 5 EXCERPTS
    CITES METHODS & BACKGROUND
    HIGHLY INFLUENCED

    Knowledge-Enriched Visual Storytelling

    VIEW 1 EXCERPT
    CITES BACKGROUND

    FILTER CITATIONS BY YEAR

    2018
    2020

    CITATION STATISTICS

    • 4 Highly Influenced Citations

    • Averaged 9 Citations per year from 2018 through 2020

    References

    Publications referenced by this paper.
    SHOWING 1-10 OF 64 REFERENCES

    Video Captioning via Hierarchical Reinforcement Learning

    VIEW 11 EXCERPTS
    HIGHLY INFLUENTIAL

    Video Paragraph Captioning Using Hierarchical Recurrent Neural Networks

    VIEW 1 EXCERPT

    Deep Reinforcement Learning-Based Image Captioning with Embedding Reward

    Jointly Modeling Embedding and Translation to Bridge Video and Language

    A Hierarchical Approach for Generating Descriptive Image Paragraphs

    VIEW 2 EXCERPTS

    Deep Visual-Semantic Alignments for Generating Image Descriptions

    VIEW 2 EXCERPTS