Query and Output: Generating Words by Querying Distributed Word Representations for Paraphrase Generation

@inproceedings{Ma2018QueryAO,
  title={Query and Output: Generating Words by Querying Distributed Word Representations for Paraphrase Generation},
  author={Shuming Ma and Xu Sun and Wei Li and Sujian Li and Wenjie Li and Xuancheng Ren},
  booktitle={NAACL},
  year={2018}
}
Most recent approaches use the sequence-to-sequence model for paraphrase generation. The existing sequence-to-sequence model tends to memorize the words and the patterns in the training dataset instead of learning the meaning of the words. Therefore, the generated sentences are often grammatically correct but semantically improper. In this work, we introduce a novel model based on the encoder-decoder framework, called Word Embedding Attention Network (WEAN). Our proposed model generates the… 

Figures and Tables from this paper

User-Oriented Paraphrase Generation With Keywords Controlled Network
TLDR
A keyword controlled network (KCN) which can be used as an assistant paraphrase generation tool and outperforms baselines in the automatic evaluation and the generated paraphrases meet user expectation in the human evaluation.
Abstractive Summarization with Keyword and Generated Word Attention
TLDR
A novel keyword and generated word attention model that outperforms baselines and achieves the state-of-the-art performance for abstractive summarization of natural language processing field.
Combining Variational Sampling and Metropolis--Hastings Sampling for Paraphrase Generation
TLDR
An unsupervised paraphrasing technique that works in word-level as well as phrase-level for sentence-level paraphrase generation and improves the quality of paraphrases when compared with techniques paraphrase by direct wordlevel sampling only is proposed.
Regularizing Output Distribution of Abstractive Chinese Social Media Text Summarization for Improved Semantic Consistency
TLDR
This article proposes a regularization approach for the sequence-to-sequence model and makes use of what the model has learned to regularize the learning objective to alleviate the effect of the problem.
Augmenting Paraphrase Generation with Syntax Information Using Graph Convolutional Networks
TLDR
This work demonstrates the utility of syntactic information in neural paraphrase generation under the framework of sequence-to-sequence modeling, and shows that graph convolutional network-enhanced models consistently outperform their syntax-agnostic counterparts using multiple evaluation metrics.
Automatic Generation of the Draft Procuratorial Suggestions Based on an Extractive Summarization Method: BERTSLCA
TLDR
This work proposes a novel deep contextualized embeddings-based method BERTSLCA, the first work to use the pretrained language model for extractive summarization tasks in the field of Chinese judicial litigation.
DSTC8-AVSD: Multimodal Semantic Transformer Network with Retrieval Style Word Generator
TLDR
A transformer-based architecture with an attention-based word embedding layer that generates words by querying word embeddings with the aim of considering the meaning of the words at the generation stage is proposed.
ConRPG: Paraphrase Generation using Contexts as Regularizer
TLDR
A pipelined system which consists of paraphrase candidate generation based on contextual language models, candidate filtering using scoring functions, and paraphrase model training based on the selected candidates, which significantly outperforms existing paraphrase approaches in both supervised and unsupervised setups.
Paraphrase Generation as Unsupervised Machine Translation
In this paper, we propose a new paradigm for paraphrase generation by treating the task as unsupervised machine translation (UMT) based on the assumption that there must be pairs of sentences
...
...

References

SHOWING 1-10 OF 44 REFERENCES
Joint Copying and Restricted Generation for Paraphrase
TLDR
A novel Seq2Seq model to fuse a copying decoder and a restricted generative decoder that outperforms the state-of-the-art approaches in terms of both informativeness and language quality.
Abstractive Text Summarization using Sequence-to-sequence RNNs and Beyond
TLDR
This work proposes several novel models that address critical problems in summarization that are not adequately modeled by the basic architecture, such as modeling key-words, capturing the hierarchy of sentence-to-word structure, and emitting words that are rare or unseen at training time.
Abstractive Sentence Summarization with Attentive Recurrent Neural Networks
TLDR
A conditional recurrent neural network (RNN) which generates a summary of an input sentence which significantly outperforms the recently proposed state-of-the-art method on the Gigaword corpus while performing competitively on the DUC-2004 shared task.
Sequence to Sequence Learning with Neural Networks
TLDR
This paper presents a general end-to-end approach to sequence learning that makes minimal assumptions on the sequence structure, and finds that reversing the order of the words in all source sentences improved the LSTM's performance markedly, because doing so introduced many short term dependencies between the source and the target sentence which made the optimization problem easier.
Neural Paraphrase Generation with Stacked Residual LSTM Networks
TLDR
This work is the first to explore deep learning models for paraphrase generation with a stacked residual LSTM network, where it adds residual connections between L STM layers for efficient training of deep LSTMs.
A Neural Attention Model for Abstractive Sentence Summarization
TLDR
This work proposes a fully data-driven approach to abstractive sentence summarization by utilizing a local attention-based model that generates each word of the summary conditioned on the input sentence.
Improving Semantic Relevance for Sequence-to-Sequence Learning of Chinese Social Media Text Summarization
TLDR
A Semantic Relevance Based neural model is introduced to encourage high semantic similarity between texts and summaries for Chinese social media summarization and outperforms baseline systems on a social media corpus.
Distraction-based neural networks for modeling documents
TLDR
This paper proposes neural models to train computers not just to pay attention to specific regions and content of input documents with attention models, but also distract them to traverse between different content of a document so as to better grasp the overall meaning for summarization.
Table-to-text Generation by Structure-aware Seq2seq Learning
TLDR
The attention visualizations and case studies show that the novel structure-aware seq2seq architecture which consists of field-gating encoder and description generator with dual attention is capable of generating coherent and informative descriptions based on the comprehensive understanding of both the content and the structure of a table.
Neural Summarization by Extracting Sentences and Words
TLDR
This work develops a general framework for single-document summarization composed of a hierarchical document encoder and an attention-based extractor that allows for different classes of summarization models which can extract sentences or words.
...
...