Abstractive Text Summarization using Sequence-to-sequence RNNs and Beyond

  title={Abstractive Text Summarization using Sequence-to-sequence RNNs and Beyond},
  author={Ramesh Nallapati and Bowen Zhou and C{\'i}cero Nogueira dos Santos and Çaglar G{\"u}lçehre and Bing Xiang},
  booktitle={Conference on Computational Natural Language Learning},
In this work, we model abstractive text summarization using Attentional Encoder-Decoder Recurrent Neural Networks, and show that they achieve state-of-the-art performance on two different corpora. We propose several novel models that address critical problems in summarization that are not adequately modeled by the basic architecture, such as modeling key-words, capturing the hierarchy of sentence-to-word structure, and emitting words that are rare or unseen at training time. Our work shows that… 

Figures and Tables from this paper

SummaRuNNer: A Recurrent Neural Network Based Sequence Model for Extractive Summarization of Documents

We present SummaRuNNer, a Recurrent Neural Network (RNN) based sequence model for extractive summarization of documents and show that it achieves performance better than or comparable to

Augmenting Neural Sentence Summarization Through Extractive Summarization

A novel approach to improve neural sentence summarization by using extractive summarization, which aims at taking full advantage of the document information as much as possible and presents both of streamline strategy and system combination strategy to achieve the fusion of the contents in different views.

Topic Augmented Generator for Abstractive Summarization

This paper proposes a new decoder where the output summary is generated by conditioning on both the input text and the latent topics of the document, and achieves strongly improved ROUGE scores when compared to state-of-the-art models.

Improving Abstractive Text Summarization with History Aggregation

An aggregation mechanism based on the Transformer model is proposed to address the challenge of long text representation and can review history information to make encoder hold more memory capacity.

Improving Abstractive Summarization via Dilated Convolution

A sequence-to-sequence based hybrid neural network model is proposed for abstractive summarization that outperforms several strong baselines on both of Gigawords corpus and DUC-2004 task.

Abstractive summarization by neural attention model with document content memory

A generative approach for abstractive summarization, which creates summaries based on a language model by reflecting the key concepts of the original document and the characteristics of summaries, is proposed.

Abstractive Document Summarization via Neural Model with Joint Attention

This work proposes to add an attention mechanism on output sequence to avoid repetitive contents and use the subword method to deal with the rare and unknown words in document summarization.

Sentence Selective Neural Extractive Summarization with Reinforcement Learning

  • Laifu ChenM. Nguyen
  • Computer Science
    2019 11th International Conference on Knowledge and Systems Engineering (KSE)
  • 2019
A sentence level selective encoding mechanism to select important feature before extracting sentences is developed, and a novel reinforcement learning based training algorithm is used to extend the sequence model.

Abstractive Summarization Using Attentive Neural Techniques

This work modify and optimize a translation model with self-attention for generating abstractive sentence summaries, and proposes a new approach based on the intuition that an abstractive model requires an Abstractive evaluation.

Abstractive Text Summarization via Stacked LSTM*

This work utilizes a more complex feature representation for summarization via stacked LSTM to achieve a better performance compared to the existing state-of-the-art phrase-based system on the task of text summarization on gigaword dataset.



A Neural Attention Model for Abstractive Sentence Summarization

This work proposes a fully data-driven approach to abstractive sentence summarization by utilizing a local attention-based model that generates each word of the summary conditioned on the input sentence.

Sequence-to-Sequence RNNs for Text Summarization

This work casts text summarization as a sequence-to-sequence problem and applies the attentional encoder-decoder RNN that has been shown to be successful for Machine Translation and significantly outperforms the state-of-the art model of Rush et al. (2015).

Neural Summarization by Extracting Sentences and Words

This work develops a general framework for single-document summarization composed of a hierarchical document encoder and an attention-based extractor that allows for different classes of summarization models which can extract sentences or words.

Abstractive Sentence Summarization with Attentive Recurrent Neural Networks

A conditional recurrent neural network (RNN) which generates a summary of an input sentence which significantly outperforms the recently proposed state-of-the-art method on the Gigaword corpus while performing competitively on the DUC-2004 shared task.

HEADS: Headline Generation as Sequence Prediction Using an Abstract Feature-Rich Space

This study presents a sequence-prediction technique for learning how editors title their news stories, and trains and tests the model on an extensive corpus of financial news, and compares it against a number of baselines.

A Hierarchical Neural Autoencoder for Paragraphs and Documents

This paper introduces an LSTM model that hierarchically builds an embedding for a paragraph from embeddings for sentences and words, then decodes this embedding to reconstruct the original paragraph and evaluates the reconstructed paragraph using standard metrics to show that neural models are able to encode texts in a way that preserve syntactic, semantic, and discourse coherence.

Extractive Summarization Using Supervised and Semi-Supervised Learning

This paper investigates co-training by combining labeled and unlabeled data and shows that this semi-supervised learning approach achieves comparable performance to its supervised counterpart and saves about half of the labeling time cost.

Temporal Attention Model for Neural Machine Translation

This work proposes a novel mechanism to address some of these limitations and improve the NMT attention that memorizes the alignments temporally and modulates the attention with the accumulated temporal memory, as the decoder generates the candidate translation.

LCSTS: A Large Scale Chinese Short Text Summarization Dataset

A large corpus of Chinese short text summarization dataset constructed from the Chinese microblogging website Sina Weibo is introduced and recurrent neural network is introduced for the summary generation and promising results are achieved.