• Corpus ID: 53046333

Hierarchical Text Generation using an Outline

@article{Drissi2018HierarchicalTG,
  title={Hierarchical Text Generation using an Outline},
  author={Mehdi Drissi and Olivia Watkins and Jugal Kumar Kalita},
  journal={ArXiv},
  year={2018},
  volume={abs/1810.08802}
}
Many challenges in natural language processing require generating text, including language translation, dialogue generation, and speech recognition. For all of these problems, text generation becomes more difficult as the text becomes longer. Current language models often struggle to keep track of coherence for long pieces of text. Here, we attempt to have the model construct and use an outline of the text it generates to keep it focused. We find that the usage of an outline improves perplexity… 
[PDF] Semantic Reader
9 Citations
Highly Influential Citations
2
Background Citations
8
Methods Citations
4

Figures and Tables from this paper

9 Citations

Learning to Generate Explainable Plots for Neural Story Generation

A latent variable model is proposed for neural story generation that treats an outline, which is a natural language sentence explainable to humans, as a latent variable to represent a high-level plot that bridges the input and output.

Learning to Predict Explainable Plots for Neural Story Generation

A latent variable model is proposed for neural story generation that treats an outline, which is a natural language sentence explainable to humans, as a latent variable to represent a high-level plot that bridges the input and output.

Facts2Story: Controlling Text Generation by Key Facts

A plan-and-cloze model is proposed (using fine-tuned XLNet) which produces competitive fluency while adhering to the requested content, and it is shown that while auto-regressive, unidirectional Language Models such as GPT2 produce better fluency, they struggle to adhere to the demanded facts.

Neural Language Generation: Formulation, Methods, and Evaluation

There is no standard way to assess the quality of text produced by these generative models, which constitutes a serious bottleneck towards the progress of the field, so this survey will provide an informative overview of formulations, methods, and assessments of neural natural language generation.

High Quality Real-Time Structured Debate Generation

The ability to generate debates in real-time on complex topics at a quality that is close to humans, as evaluated by the style, content, and strategy metrics used for judging competitive human debates is demonstrated.

AS DISCRETE LATENT VARIABLES

It is found that extracting words as latent variables significantly outperforms the state-of-the-art discrete latent variable models such as VQ-VAE and there is a hierarchy in language such that an entire text can be predicted much more easily based on a sequence of a small number of keywords, which can be easily found by classical methods as tf-idf.

A Survey of the Usages of Deep Learning for Natural Language Processing

An introduction to the field and a quick overview of deep learning architectures and methods is provided and a discussion of the current state of the art is provided along with recommendations for future research in the field.

Open Domain Event Text Generation

A novel random drop component is proposed that randomly deletes words from the retrieved sentences making the model more robust for handling long input sentences and the experimental results show that the carefully-designed architecture does help generate better event text.

Extractive Summary as Discrete Latent Variables

It is found that extracting tokens as latent variables significantly outperforms the state-of-the-art discrete latent variable models such as VQ-VAE and is speculated that this extraction process may be useful for unsupervised hierarchical text generation.

References

SHOWING 1-10 OF 23 REFERENCES

Generating Wikipedia by Summarizing Long Sequences

It is shown that generating English Wikipedia articles can be approached as a multi- document summarization of source documents and a neural abstractive model is introduced, which can generate fluent, coherent multi-sentence paragraphs and even whole Wikipedia articles.

Hierarchical Neural Story Generation

This work collects a large dataset of 300K human-written stories paired with writing prompts from an online forum that enables hierarchical story generation, where the model first generates a premise, and then transforms it into a passage of text.

Globally Coherent Text Generation with Neural Checklist Models

The neural checklist model is presented, a recurrent neural network that models global coherence by storing and updating an agenda of text strings which should be mentioned somewhere in the output, and demonstrates high coherence with greatly improved semantic coverage of the agenda.

Text Summarization

  • Dou Shen
  • Computer Science
    Encyclopedia of Database Systems
  • 2009
This thesis deals with the development of a new text summarization method that uses the latent semantic analysis (lsa). The language-independent analysis is able to capture interrelationships among…

Cold Fusion: Training Seq2Seq Models Together with Language Models

It is shown that Seq2Seq models with Cold Fusion are able to better utilize language information enjoying i) faster convergence and better generalization, and ii) almost complete transfer to a new domain while using less than 10% of the labeled training data.

Coarse-to-Fine Attention Models for Document Summarization

A novel coarse-to-fine attention model that hierarchically reads a document, using coarse attention to select top-level chunks of text and fine attention to read the words of the chosen chunks, which achieves the desired behavior of sparsely attending to subsets of the document for generation.

Linguistic Features of Helpfulness in Automated Support for Creative Writing

An emerging NLP application that supports creative writing by automatically suggesting continuing sentences in a story is examined, and the task of predicting helpfulness based on automatically detected linguistic features of the suggestions is explored.

Neural Machine Translation by Jointly Learning to Align and Translate

It is conjecture that the use of a fixed-length vector is a bottleneck in improving the performance of this basic encoder-decoder architecture, and it is proposed to extend this by allowing a model to automatically (soft-)search for parts of a source sentence that are relevant to predicting a target word, without having to form these parts as a hard segment explicitly.

Sequence to Sequence Learning with Neural Networks

This paper presents a general end-to-end approach to sequence learning that makes minimal assumptions on the sequence structure, and finds that reversing the order of the words in all source sentences improved the LSTM's performance markedly, because doing so introduced many short term dependencies between the source and the target sentence which made the optimization problem easier.

Text Summarization Techniques: A Brief Survey

The main approaches to automatic text summarization are described and the effectiveness and shortcomings of the different methods are described.