Abstractive Summarization of Reddit Posts with Multi-level Memory Networks

@article{Kim2019AbstractiveSO,
  title={Abstractive Summarization of Reddit Posts with Multi-level Memory Networks},
  author={Byeongchang Kim and Hyunwoo Kim and Gunhee Kim},
  journal={ArXiv},
  year={2019},
  volume={abs/1811.00783}
}
We address the problem of abstractive summarization in two directions: proposing a novel dataset and a new model. [] Key Method Second, we propose a novel abstractive summarization model named multi-level memory networks (MMN), equipped with multi-level memory to store the information of text from different levels of abstraction. With quantitative evaluation and user studies via Amazon Mechanical Turk, we show the Reddit TIFU dataset is highly abstractive and the MMN outperforms the state-of-the-art…

Figures and Tables from this paper

Neural Abstractive Unsupervised Summarization of Online News Discussions
TLDR
A novel method that generates abstractive summaries of online news discussions that represents the most relevant aspects of a news item that users comment on, incorporating the social context as a source of information to summarize texts in online social networks.
Transformer-Based Abstractive Summarization for Reddit and Twitter: Single Posts vs. Comment Pools in Three Languages
TLDR
It is shown that a ‘one-for-all’ neural-network summarization model is still impossible to reach, while fine-tuning for platform affordances works well and that fine- Tuned T5 works best for small-scale social media data, but LongFormer is helpful for larger-scale pool summarizations.
TLDR9+: A Large Scale Resource for Extreme Summarization of Social Media Posts
TLDR
This paper introduces TLDR9+ –a large-scale summarization dataset– containing over 9 million training instances extracted from Reddit discussion forum ([HTTP]).
Enriching and Controlling Global Semantics for Text Summarization
TLDR
This paper introduces a neural topic model empowered with normalizing flow to capture the global semantics of the document, which are then integrated into the summarization model, and introduces a mechanism to control the amount of global semantics supplied to the text generation module.
ONLINE NEWS DISCUSSIONS
TLDR
A novel method that generates abstractive summaries of online news discussions that represents the most relevant aspects of a news item that users comment on, incorporating the social context as a source of information to summarize texts in online social networks.
About ? Extreme Summarization with Topic-Aware Convolutional Neural Networks
  • Shashi Narayan
  • Computer Science
  • 2019
TLDR
This work introduces extreme summarization, a new single-document summarization task which aims at creating a short, one-sentence news summary answering the question “What is the article about?” and proposes a novel abstractive model which is conditioned on the article’s topics and based entirely on convolutional neural networks.
Exploring Domain Shift in Extractive Text Summarization
TLDR
This paper extends the conventional definition of the domain from categories into data sources for the text summarization task, then re-purpose a multi-domain summarization dataset and verifies how the gap between different domains influences the performance of neural summarization models.
PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization
TLDR
This work proposes pre-training large Transformer-based encoder-decoder models on massive text corpora with a new self-supervised objective, PEGASUS, and demonstrates it achieves state-of-the-art performance on all 12 downstream datasets measured by ROUGE scores.
Neural Abstractive Text Summarization with Sequence-to-Sequence Models
TLDR
This article provides a comprehensive literature survey on different seq2seq models for abstractive text summarization from the viewpoint of network structures, training strategies, and summary generation algorithms.
SummaReranker: A Multi-Task Mixture-of-Experts Re-ranking Framework for Abstractive Summarization
TLDR
It is shown that it is possible to directly train a second-stage model performing re-ranking on a set of summary candidates, and a mixture-of-experts SummaReranker learns to select a better candidate and consistently improves the performance of the base model.
...
...

References

SHOWING 1-10 OF 53 REFERENCES
Abstractive Document Summarization with a Graph-Based Attentional Neural Model
TLDR
A novel graph-based attention mechanism in the sequence-to-sequence framework to address the saliency factor of summarization, which has been overlooked by prior works and is competitive with state-of-the-art extractive methods.
Hybrid MemNet for Extractive Summarization
TLDR
A fully data-driven end-to-end deep network which is called as Hybrid MemNet for single document summarization task, which learns the continuous unified representation of a document before generating its summary.
A Unified Model for Extractive and Abstractive Summarization using Inconsistency Loss
TLDR
By end-to-end training the model with the inconsistency loss and original losses of extractive and abstractive models, the model achieves state-of-the-art ROUGE scores while being the most informative and readable summarization on the CNN/Daily Mail dataset in a solid human evaluation.
Abstractive Text Summarization using Sequence-to-sequence RNNs and Beyond
TLDR
This work proposes several novel models that address critical problems in summarization that are not adequately modeled by the basic architecture, such as modeling key-words, capturing the hierarchy of sentence-to-word structure, and emitting words that are rare or unseen at training time.
A Deep Reinforced Model for Abstractive Summarization
TLDR
A neural network model with a novel intra-attention that attends over the input and continuously generated output separately, and a new training method that combines standard supervised word prediction and reinforcement learning (RL) that produces higher quality summaries.
Attend to You: Personalized Image Captioning with Context Sequence Memory Networks
TLDR
This work proposes a novel captioning model named Context Sequence Memory Network (CSMN), and shows the effectiveness of the three novel features of CSMN and its performance enhancement for personalized image captioning over state-of-the-art captioning models.
Bottom-Up Abstractive Summarization
TLDR
This work explores the use of data-efficient content selectors to over-determine phrases in a source document that should be part of the summary, and shows that this approach improves the ability to compress text, while still generating fluent summaries.
Get To The Point: Summarization with Pointer-Generator Networks
TLDR
A novel architecture that augments the standard sequence-to-sequence attentional model in two orthogonal ways, using a hybrid pointer-generator network that can copy words from the source text via pointing, which aids accurate reproduction of information, while retaining the ability to produce novel words through the generator.
Don’t Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization
TLDR
A novel abstractive model is proposed which is conditioned on the article’s topics and based entirely on convolutional neural networks, outperforming an oracle extractive system and state-of-the-art abstractive approaches when evaluated automatically and by humans.
Content Selection in Deep Learning Models of Summarization
TLDR
It is suggested that it is easier to create a summarizer for a new domain than previous work suggests and the benefit of deep learning models for summarization for those domains that do have massive datasets is brought into question.
...
...