Corpus ID: 211259070

Discriminative Adversarial Search for Abstractive Summarization

@article{Scialom2020DiscriminativeAS,
  title={Discriminative Adversarial Search for Abstractive Summarization},
  author={Thomas Scialom and Paul-Alexis Dray and Sylvain Lamprier and Benjamin Piwowarski and Jacopo Staiano},
  journal={ArXiv},
  year={2020},
  volume={abs/2002.10375}
}
We introduce a novel approach for sequence decoding, Discriminative Adversarial Search (DAS), which has the desirable properties of alleviating the effects of exposure bias without requiring external metrics. Inspired by Generative Adversarial Networks (GANs), wherein a discriminator is used to improve the generator, our method differs from GANs in that the generator parameters are not updated at training time and the discriminator is only used to drive sequence generation at inference time… Expand

Figures and Tables from this paper

ColdGANs: Taming Language GANs with Cautious Sampling Strategies
TLDR
For the first time, to the best of the knowledge, the proposed language GANs compare favorably to MLE, and obtain improvements over the state-of-the-art on three generative tasks, namely unconditional text generation, question generation, and abstractive summarization. Expand
Adding A Filter Based on The Discriminator to Improve Unconditional Text Generation
TLDR
A novel mechanism by adding a filter which has the same input as the discriminator, which consistently outperforms the ALMs and all kinds of GANs across two benchmark data sets. Expand
Using Pre-Trained Transformer for Better Lay Summarization
TLDR
This paper presents the approach of using Pre-training with Extracted Gap-sentences for Abstractive Summarization to produce the lay summary and combining those with the extractive summarization model using Bidirectional Encoder Representations from Transformers and readability metrics that measure the readability of the sentence to further improve the quality of the summary. Expand
Toward Stance-based Personas for Opinionated Dialogues
TLDR
This work introduces a novel dataset allowing to explore different stance-based persona representations and their impact on claim generation, showing that they are able to grasp abstract and profound aspects of the author persona. Expand
To Beam Or Not To Beam: That is a Question of Cooperation for Language GANs
TLDR
This paper shows that the SelfGAN framework, built on this cooperative principle, outperforms Teacher Forcing and obtains state-of-the-art results on two challenging tasks, Summarization and Question Generation. Expand
BEAMetrics: A Benchmark for Language Generation Evaluation Evaluation
TLDR
The Benchmark to Evaluate Automatic Metrics (BEAMetrics) is described, a resource to make research into new metrics itself easier to evaluate and facilitate research into better metrics – particularly those that can account for the complex interaction between context and general knowledge inherent to many modern NLP applications. Expand
Event Graph based Sentence Fusion
Sentence fusion is a conditional generation task that merges several related sentences into a coherent one, which can be deemed as a summary sentence. The importance of sentence fusion has long beenExpand
Generating texts under constraint through discriminator-guided MCTS
TLDR
This paper proposes several original methods to search this generation tree, notably the Monte Carlo Tree Search (MCTS) which provides theoretical guarantees on the search efficiency, but also simpler methods based on re-ranking a pool of diverse sequences using the discriminator scores. Expand
Neural Text Generation with Part-of-Speech Guided Softmax
TLDR
This work proposes using linguistic annotation, i.e., part-of-speech (POS), to guide the text generation using POS Guided Softmax (POSG-Softmax) to explicitly model two posterior probabilities: (i) next-POS, and (ii) nexttoken from the vocabulary of the target POS. Expand
Rethinking Automatic Evaluation in Sentence Simplification
TLDR
This paper first proposes a simple adaptation of QUESTEVAL to Sentence Simplification tasks, then extensively evaluates the correlations w.r.t. human judgement for several metrics and shows that BERTScore and QUESTevAL obtains the best results. Expand

References

SHOWING 1-10 OF 38 REFERENCES
Self-Adversarial Learning with Comparative Discrimination for Text Generation
TLDR
This work proposes a novel self-adversarial learning (SAL) paradigm for improving GANs' performance in text generation that employs a comparative discriminator which is a pairwise classifier for comparing the text quality between a pair of samples. Expand
Language GANs Falling Short
TLDR
The impact of exposure bias on sample quality is less severe than previously thought, and temperature tuning provides a better quality / diversity trade-off than adversarial training while being easier to train, easier to cross-validate, and less computationally expensive. Expand
Adding A Filter Based on The Discriminator to Improve Unconditional Text Generation
TLDR
A novel mechanism by adding a filter which has the same input as the discriminator, which consistently outperforms the ALMs and all kinds of GANs across two benchmark data sets. Expand
A Deep Reinforced Model for Abstractive Summarization
TLDR
A neural network model with a novel intra-attention that attends over the input and continuously generated output separately, and a new training method that combines standard supervised word prediction and reinforcement learning (RL) that produces higher quality summaries. Expand
Generative Adversarial Nets
We propose a new framework for estimating generative models via an adversarial process, in which we simultaneously train two models: a generative model G that captures the data distribution, and aExpand
ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators
TLDR
The contextual representations learned by the proposed replaced token detection pre-training task substantially outperform the ones learned by methods such as BERT and XLNet given the same model size, data, and compute. Expand
Cooperative Generator-Discriminator Networks for Abstractive Summarization with Narrative Flow
TLDR
To promote research toward abstractive summarization with narrative flow, a new dataset is introduced, Scientific Abstract SummarieS (SASS), where the abstracts are used as proxy gold summaries for scientific articles and Co-opNet is proposed, a novel transformer-based framework where the generator works with the discourse discriminator to compose a long-form summary. Expand
Get To The Point: Summarization with Pointer-Generator Networks
TLDR
A novel architecture that augments the standard sequence-to-sequence attentional model in two orthogonal ways, using a hybrid pointer-generator network that can copy words from the source text via pointing, which aids accurate reproduction of information, while retaining the ability to produce novel words through the generator. Expand
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
TLDR
A new language representation model, BERT, designed to pre-train deep bidirectional representations from unlabeled text by jointly conditioning on both left and right context in all layers, which can be fine-tuned with just one additional output layer to create state-of-the-art models for a wide range of tasks. Expand
Attention is All you Need
TLDR
A new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely is proposed, which generalizes well to other tasks by applying it successfully to English constituency parsing both with large and limited training data. Expand
...
1
2
3
4
...