Hierarchical Neural Story Generation

@inproceedings{Fan2018HierarchicalNS,
  title={Hierarchical Neural Story Generation},
  author={Angela Fan and Mike Lewis and Yann Dauphin},
  booktitle={Annual Meeting of the Association for Computational Linguistics},
  year={2018}
}
We explore story generation: creative systems that can build coherent and fluent passages of text about a topic. [] Key Method We gain further improvements with a novel form of model fusion that improves the relevance of the story to the prompt, and adding a new gated multi-scale self-attention mechanism to model long-range context. Experiments show large improvements over strong baselines on both automated and human evaluations. Human judges prefer stories generated by our approach to those from a strong…
[PDF] Semantic Reader
846 Citations
Highly Influential Citations
183
Background Citations
442
Methods Citations
431
Results Citations
16

846 Citations

Cue Me In: Content-Inducing Approaches to Interactive Story Generation

This work focuses on the task of interactive story generation, where the user provides the model mid-level sentence abstractions in the form of cue phrases during the generation process, and presents two content-inducing approaches to effectively incorporate this additional information.

Learning to Generate Explainable Plots for Neural Story Generation

A latent variable model is proposed for neural story generation that treats an outline, which is a natural language sentence explainable to humans, as a latent variable to represent a high-level plot that bridges the input and output.

Learning to Predict Explainable Plots for Neural Story Generation

A latent variable model is proposed for neural story generation that treats an outline, which is a natural language sentence explainable to humans, as a latent variable to represent a high-level plot that bridges the input and output.

Collaborative Storytelling with Large-scale Neural Language Models

This paper presents a collaborative storytelling system which works with a human storyteller to create a story by generating new utterances based on the story so far and proposes a sample-and-rank approach to improve utterance quality.

Content Planning for Neural Story Generation with Aristotelian Rescoring

This work uses a plot-generation language model along with an ensemble of rescoring models that each implement an aspect of good story-writing as detailed in Aristotle's Poetics to present a system that focuses on how to learn good plot structures to guide story generation.

EtriCA: Event-Triggered Context-Aware Story Generation Augmented by Cross Attention

EtriCA is presented, a novel neural generation model, which improves the relevance and coherence of the generated stories through residually mapping context features to event sequences with a cross-attention mechanism.

A Temporal Variational Model for Story Generation

Experiments with a latent vector planning approach based on a TD-VAE (Temporal Difference Variational Autoencoder) using the model for conditioning and reranking for text generation demonstrate strong performance in automatic cloze and swapping evaluations.

Automated Story Generation as Question-Answering

A novel approach to automated story generation that treats the problem as one of generative question-answering, and shows that stories generated by the system are 15% more coherent than those generated by BART fine-tuned on a story corpus to generate backward from a given ending.

Designing an Automatic Story Evaluation Metric

This paper proposes a novel framework for such an automatic story metric using emotional states and a commonsense model to measure the coherence and meaning-making ability of a story.

A Knowledge-Enhanced Pretraining Model for Commonsense Story Generation

A knowledge-enhanced pretraining model to utilize commonsense knowledge from external knowledge bases to generate reasonable stories that can generate more reasonable stories than state-of-the-art baselines, particularly in terms of logic and global coherence.
...

References

SHOWING 1-10 OF 30 REFERENCES

Event Representations for Automated Story Generation with Deep Neural Nets

The question of event representations that provide a mid-level of abstraction between words and sentences in order to retain the semantic information of the original data while minimizing event sparsity is explored.

Story Generation from Sequence of Independent Short Descriptions

A deep recurrent neural network architecture is implemented that encodes sequence of variable length input descriptions to corresponding latent representations and decodes them to produce well formed comprehensive story like summaries.

A Hierarchical Neural Autoencoder for Paragraphs and Documents

This paper introduces an LSTM model that hierarchically builds an embedding for a paragraph from embeddings for sentences and words, then decodes this embedding to reconstruct the original paragraph and evaluates the reconstructed paragraph using standard metrics to show that neural models are able to encode texts in a way that preserve syntactic, semantic, and discourse coherence.

Generating Long and Diverse Responses with Neural Conversation Models

This work presents new training and decoding methods that improve the quality, coherence, and diversity of long responses generated using sequence-to-sequence models, and introduces a stochastic beam-search algorithm with segment-by-segment reranking which lets us inject diversity earlier in the generation process.

A Neural Attention Model for Abstractive Sentence Summarization

This work proposes a fully data-driven approach to abstractive sentence summarization by utilizing a local attention-based model that generates each word of the summary conditioned on the input sentence.

Hierarchical Text Generation and Planning for Strategic Dialogue

An approach to learning representations of messages in dialogues by maximizing the likelihood of subsequent sentences and actions, which decouples the semantics of the dialogue utterance from its linguistic realization, which outperforms previous work both linguistically and strategically.

Generating Wikipedia by Summarizing Long Sequences

It is shown that generating English Wikipedia articles can be approached as a multi- document summarization of source documents and a neural abstractive model is introduced, which can generate fluent, coherent multi-sentence paragraphs and even whole Wikipedia articles.

Challenges in Data-to-Document Generation

A new, large-scale corpus of data records paired with descriptive documents is introduced, a series of extractive evaluation methods for analyzing performance are proposed, and baseline results are obtained using current neural generation methods.

Toward Automated Story Generation with Markov Chain Monte Carlo Methods and Deep Neural Networks

An approach to automated story generation using Markov Chain Monte Carlo (MCMC) sampling, which uses a sampling algorithm based on Metropolis-Hastings to generate a probability distribution which can be used to generate stories via random sampling that adhere to criteria learned by recurrent neural networks.

A Diversity-Promoting Objective Function for Neural Conversation Models

This work proposes using Maximum Mutual Information (MMI) as the objective function in neural models, and demonstrates that the proposed MMI models produce more diverse, interesting, and appropriate responses, yielding substantive gains in BLEU scores on two conversational datasets and in human evaluations.