Corpus ID: 60440448

Insertion Transformer: Flexible Sequence Generation via Insertion Operations

@inproceedings{Stern2019InsertionTF,
  title={Insertion Transformer: Flexible Sequence Generation via Insertion Operations},
  author={Mitchell Stern and William Chan and J. Kiros and Jakob Uszkoreit},
  booktitle={ICML},
  year={2019}
}
  • Mitchell Stern, William Chan, +1 author Jakob Uszkoreit
  • Published in ICML 2019
  • Computer Science, Mathematics
  • We present the Insertion Transformer, an iterative, partially autoregressive model for sequence generation based on insertion operations. [...] Key Method This flexibility confers a number of advantages: for instance, not only can our model be trained to follow specific orderings such as left-to-right generation or a binary tree traversal, but it can also be trained to maximize entropy over all valid insertions for robustness.Expand Abstract

    Paper Mentions

    Levenshtein Transformer
    40
    Insertion-based Decoding with Automatically Inferred Generation Order
    39
    Insertion-Deletion Transformer
    3
    An Empirical Study of Generation Order for Machine Translation
    5
    Felix: Flexible Text Editing Through Tagging and Insertion
    1
    Big Bidirectional Insertion Representations for Documents
    3
    XL-Editor: Post-editing Sentences with XLNet
    2

    References

    Publications referenced by this paper.
    SHOWING 1-10 OF 30 REFERENCES
    Insertion-based Decoding with Automatically Inferred Generation Order
    39
    Blockwise Parallel Decoding for Deep Autoregressive Models
    25
    Attention is All you Need
    10513
    Semi-Autoregressive Neural Machine Translation
    19
    Non-Autoregressive Neural Machine Translation
    138
    Sequence-Level Knowledge Distillation
    219
    Sequence Transduction with Recurrent Neural Networks
    479
    The Importance of Generation Order in Language Modeling
    16
    Neural Machine Translation by Jointly Learning to Align and Translate
    12355
    Sequence to Sequence Learning with Neural Networks
    10133