Corpus ID: 49669485

Latent Alignment and Variational Attention

  title={Latent Alignment and Variational Attention},
  author={Y. Deng and Yoon Kim and Justin T Chiu and Demi Guo and Alexander M. Rush},
  • Y. Deng, Yoon Kim, +2 authors Alexander M. Rush
  • Published 2018
  • Computer Science, Mathematics
  • ArXiv
  • Neural attention has become central to many state-of-the-art models in natural language processing and related domains. Attention networks are an easy-to-train and effective method for softly simulating alignment; however, the approach does not marginalize over latent alignments in a probabilistic sense. This property makes it difficult to compare attention to other alignment approaches, to compose it with probabilistic models, and to perform posterior inference conditioned on observed data. A… CONTINUE READING
    60 Citations
    Bayesian Attention Modules
    • Highly Influenced
    • PDF
    Discrete Variational Attention Models for Language Generation
    Amortized Context Vector Inference for Sequence-to-Sequence Networks
    • PDF
    Investigation of Transformer-based Latent Attention Models for Neural Machine Translation
    • PDF
    Exploring A Zero-Order Direct Hmm Based on Latent Attention for Automatic Speech Recognition
    • 1
    • PDF
    Understanding Multi-Head Attention in Abstractive Summarization
    • 3
    • PDF


    Learning Hard Alignments with Variational Inference
    • 23
    • PDF
    Structured Attention Networks
    • 232
    • PDF
    A Regularized Framework for Sparse and Structured Neural Attention
    • 57
    • PDF
    Autoencoding Variational Inference For Topic Models
    • 182
    • PDF
    Structured Attentions for Visual Question Answering
    • 71
    • PDF
    Importance Weighted Autoencoders
    • 694
    • PDF
    Attention is All you Need
    • 15,791
    • Highly Influential
    • PDF
    Variational Recurrent Neural Machine Translation
    • 43
    • PDF