Complexity-Weighted Loss and Diverse Reranking for Sentence Simplification

@inproceedings{Kriz2019ComplexityWeightedLA,
  title={Complexity-Weighted Loss and Diverse Reranking for Sentence Simplification},
  author={Reno Kriz and Jo{\~a}o Sedoc and Marianna Apidianaki and Carolina Zheng and G. Kumar and Eleni Miltsakaki and Chris Callison-Burch},
  booktitle={NAACL},
  year={2019}
}
Sentence simplification is the task of rewriting texts so they are easier to understand. Recent research has applied sequence-to-sequence (Seq2Seq) models to this task, focusing largely on training-time improvements via reinforcement learning and memory augmentation. One of the main problems with applying generic Seq2Seq models for simplification is that these models tend to copy directly from the original sentence, resulting in outputs that are relatively long and complex. We aim to alleviate… Expand
Semi-Supervised Text Simplification with Back-Translation and Asymmetric Denoising Autoencoders
TLDR
This work investigates how to leverage large amounts of unpaired corpora in TS task and proposes asymmetric denoising methods for sentences with separate complexity, which can perform competitively with multiple state-of-the-art simplification systems. Expand
Neural CRF Model for Sentence Alignment in Text Simplification
TLDR
A novel neural CRF alignment model is proposed which not only leverages the sequential nature of sentences in parallel documents but also utilizes a neural sentence pair model to capture semantic similarity. Expand
Sentence Simplification Based on Multi-Stage Encoder Model
TLDR
A multi-stage encoder based Seq2seq model for sentence simplification that outperforms state-of-the-art baseline simplification systems and introduces a novel attention connection method which could help the decoder to make full use of the information of encoder. Expand
Neural CRF Sentence Alignment Model for Text Simplification
The success of a text simplification system heavily depends on the quality and quantity of complex-simple sentence pairs in the training corpus, which are extracted by aligning sentences betweenExpand
Zero-Shot Crosslingual Sentence Simplification
TLDR
A zero-shot modeling framework which transfers simplification knowledge from English to another language (for which no parallel simplification corpus exists) while generalizing across languages and tasks is proposed. Expand
On the Helpfulness of Document Context to Sentence Simplification
TLDR
This paper is the first to investigate the helpfulness of document context on sentence simplification and apply it to the sequence-to-sequence model and proposes a new model that makes full use of the context information. Expand
Improving Human Text Simplification with Sentence Fusion
TLDR
A graph-based sentence fusion approach to augment human simplification and a reranking approach to both select high quality simplifications and to allow for targeting simplifications with varying levels of simplicity are introduced. Expand
Extremely Low Resource Text simplification with Pre-trained Transformer Language Model
TLDR
A simple approach which fine-tunes the pre-trained language model for text simplification with a small parallel corpus and shows that TransformerLM, which is a simple text generation model, substantially outperforms a strong baseline. Expand
HTSS: A novel hybrid text summarisation and simplification architecture
TLDR
This work extends the well-known pointer generator model for the combined task of summarisation and simplification using a novel hybrid architecture of abstractive and extractive summarisation called HTSS and shows that the model outperforms NTS and ATS on the joint task of simplification and summarisation. Expand
Comparison of Diverse Decoding Methods from Conditional Language Models
TLDR
This work performs an extensive survey of decoding-time strategies for generating diverse outputs from a conditional language model, and presents a novel method where over-sample candidates, then use clustering to remove similar sequences, thus achieving high diversity without sacrificing quality. Expand
...
1
2
3
4
...

References

SHOWING 1-10 OF 48 REFERENCES
Integrating Transformer and Paraphrase Rules for Sentence Simplification
TLDR
A novel model based on a multi-layer and multi-head attention architecture and two innovative approaches to integrate the Simple PPDB (A Paraphrase Database for Simplification), an external paraphrase knowledge base for simplification that covers a wide range of real-world simplification rules. Expand
Sentence Simplification with Deep Reinforcement Learning
TLDR
This work addresses the simplification problem with an encoder-decoder model coupled with a deep reinforcement learning framework, and explores the space of possible simplifications while learning to optimize a reward function that encourages outputs which are simple, fluent, and preserve the meaning of the input. Expand
A Monolingual Tree-based Translation Model for Sentence Simplification
TLDR
A Tree-based Simplification Model (TSM) is proposed, which, to the knowledge, is the first statistical simplification model covering splitting, dropping, reordering and substitution integrally. Expand
Sentence Simplification for Semantic Role Labeling
TLDR
A general method for learning how to iteratively simplify a sentence, thus decomposing complicated syntax into small, easy-to-process pieces and achieving near-state-of-the-art performance across syntactic variation. Expand
Generating High-Quality and Informative Conversation Responses with Sequence-to-Sequence Models
TLDR
This work focuses on the single turn setting, introduces a stochastic beam-search algorithm with segment-by-segment reranking which lets us inject diversity earlier in the generation process, and proposes a practical approach, called the glimpse-model, for scaling to large datasets. Expand
Abstractive Text Summarization using Sequence-to-sequence RNNs and Beyond
TLDR
This work proposes several novel models that address critical problems in summarization that are not adequately modeled by the basic architecture, such as modeling key-words, capturing the hierarchy of sentence-to-word structure, and emitting words that are rare or unseen at training time. Expand
A Neural Conversational Model
TLDR
A simple approach to conversational modeling which uses the recently proposed sequence to sequence framework, and is able to extract knowledge from both a domain specific dataset, and from a large, noisy, and general domain dataset of movie subtitles. Expand
Sequence to Sequence Learning with Neural Networks
TLDR
This paper presents a general end-to-end approach to sequence learning that makes minimal assumptions on the sequence structure, and finds that reversing the order of the words in all source sentences improved the LSTM's performance markedly, because doing so introduced many short term dependencies between the source and the target sentence which made the optimization problem easier. Expand
Sentence Simplification by Monolingual Machine Translation
TLDR
By relatively careful phrase-based paraphrasing this model achieves similar simplification results to state-of-the-art systems, while generating better formed output, and argues that text readability metrics such as the Flesch-Kincaid grade level should be used with caution when evaluating the output of simplification systems. Expand
Learning to Simplify Sentences with Quasi-Synchronous Grammar and Integer Programming
TLDR
A data-driven model based on quasi-synchronous grammar, a formalism that can naturally capture structural mismatches and complex rewrite operations, is presented and it is shown experimentally that the method creates simplifications that significantly reduce the reading difficulty of the input, while maintaining grammaticality and preserving its meaning. Expand
...
1
2
3
4
5
...