Neural Lattice Language Models

@article{Buckman2018NeuralLL,
  title={Neural Lattice Language Models},
  author={J. Buckman and Graham Neubig},
  journal={Transactions of the Association for Computational Linguistics},
  year={2018},
  volume={6},
  pages={529-541}
}
  • J. Buckman, Graham Neubig
  • Published 2018
  • Computer Science
  • Transactions of the Association for Computational Linguistics
  • In this work, we propose a new language modeling paradigm that has the ability to perform both prediction and moderation of information flow at multiple granularities: neural lattice language models. These models construct a lattice of possible paths through a sentence and marginalize across this lattice to calculate sequence probabilities or optimize parameters. This approach allows us to seamlessly incorporate linguistic intuitions — including polysemy and the existence of multiword lexical… CONTINUE READING
    14 Citations

    Topics from this paper

    Latent Relation Language Models
    • 14
    • PDF
    Adaptive Input Representations for Neural Language Modeling
    • 116
    • PDF
    Training Hybrid Language Models by Marginalizing over Segmentations
    • 4
    • PDF
    Lattice LSTM for Chinese Sentence Representation
    • Y. Zhang, Yile Wang, Jie Yang
    • Computer Science
    • IEEE/ACM Transactions on Audio, Speech, and Language Processing
    • 2020
    • PDF
    Syntactic Inductive Biases for Natural Language Processing
    • PDF
    Lattice Transformer for Speech Translation
    • 14
    • PDF
    Neural Lattice Search for Speech Recognition
    • R. Ma, H. Li, Qi Liu, Lu Chen, K. Yu
    • Computer Science
    • ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
    • 2020
    • 1
    Incorporating Task-specific Features into Deep Models to Classify Argument Components
    • PDF
    ELMoLex: Connecting ELMo and Lexicon Features for Dependency Parsing
    • 6
    • PDF

    References

    SHOWING 1-10 OF 42 REFERENCES
    Lattice based language models
    • 17
    Exploring the Limits of Language Modeling
    • 789
    • Highly Influential
    • PDF
    Tying Word Vectors and Word Classifiers: A Loss Framework for Language Modeling
    • 281
    • Highly Influential
    • PDF
    Neural Lattice-to-Sequence Models for Uncertain Inputs
    • 45
    • PDF
    Using the Output Embedding to Improve Language Models
    • 456
    • PDF
    One billion word benchmark for measuring progress in statistical language modeling
    • 735
    • PDF
    Cross-Lingual Word Embeddings for Low-Resource Language Modeling
    • 73
    • PDF
    Pointer Sentinel Mixture Models
    • 648
    • Highly Influential
    • PDF
    Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks
    • 2,024
    • PDF