Neural semi-Markov CRF for Monolingual Word Alignment

  title={Neural semi-Markov CRF for Monolingual Word Alignment},
  author={Wuwei Lan and Chao Jiang and Wei Xu},
Monolingual word alignment is important for studying fine-grained editing operations (i.e., deletion, addition, and substitution) in text-to-text generation tasks, such as paraphrase generation, text simplification, neutralizing biased language, etc. In this paper, we present a novel neural semi-Markov CRF alignment model, which unifies word and phrase alignments through variable-length spans. We also create a new benchmark with human annotations that cover four different text genres to… 

Figures and Tables from this paper

Towards Explainable Evaluation Metrics for Natural Language Generation
This concept paper identifies key properties and proposes key goals of explainable machine translation evaluation metrics and provides a vision of future approaches to explainable evaluation metric and their evaluation.
Importance of Data and Controllability in Neural Text Simplification
This paper proposes a novel hybrid approach that leverages linguistically-motivated rules for splitting and deletion, and couples them with a neural paraphrasing model to produce varied rewriting styles, and introduces a new data augmentation method to improve the paraphrase capability of the model.


Neural CRF Model for Sentence Alignment in Text Simplification
A novel neural CRF alignment model is proposed which not only leverages the sequential nature of sentences in parallel documents but also utilizes a neural sentence pair model to capture semantic similarity.
Gappy Phrasal Alignment By Agreement
A principled and efficient phrase-to-phrase alignment model, useful in machine translation as well as other related natural language processing problems, that shows substantial improvements in both alignment quality and translation quality over word-based Hidden Markov Models, while maintaining asymptotically equivalent runtime.
A Supervised Word Alignment Method Based on Cross-Language Span Prediction Using Multilingual BERT
The proposed method significantly outperformed previous supervised and unsupervised word alignment methods without using any bitexts for pretraining and greatly improved the word alignment accuracy by adding the context of the token to the question.
Word Alignment Modeling with Context Dependent Deep Neural Network
A novel bilingual word alignment approach based on DNN (Deep Neural Network) which outperforms the HMM and IBM model 4 baselines by 2 points in F-score and generates a very compact model with much fewer parameters.
Semi-Markov Phrase-Based Monolingual Alignment
We introduce a novel discriminative model for phrase-based monolingual alignment using a semi-Markov CRF. Our model achieves stateof-the-art alignment accuracy on two phrasebased alignment datasets
A Phrase-Based Alignment Model for Natural Language Inference
The MANLI system is presented, a new NLI aligner designed to address the alignment problem, which uses a phrase-based alignment representation, exploits external lexical resources, and capitalizes on a new set of supervised training data.
A Joint Phrasal and Dependency Model for Paraphrase Alignment
A new model for monolingual alignment is presented in which the score of an alignment decomposes over both the set ofaligned phrases as well as a set of aligned dependency arcs.
Optimal and Syntactically-Informed Decoding for Monolingual Phrase-Based Alignment
This work examines a state-of-the-art structured prediction model for the alignment task which uses a phrase-based representation and is forced to decode alignments using an approximate search approach and proposes a straightforward exact decoding technique based on integer linear programming that yields order- of-magnitude improvements in decoding speed.
End-to-End Neural Word Alignment Outperforms GIZA++
This work presents the first end-to-end neural word alignment method that consistently outperforms GIZA++ on three data sets and repurposes a Transformer model trained for supervised translation to also serve as an unsupervised word alignment model in a manner that is tightly integrated and does not affect translation quality.
Recurrent Neural Networks for Word Alignment Model
A word alignment model based on a recurrent neural network (RNN), in which an unlimited alignment history is represented by recurrently connected hidden layers, which outperforms the feed-forward neural network-based model as well as the IBM Model 4 under Japanese-English and French-English word alignment tasks.