Corpus ID: 7454685

Document Context Language Models

@article{Ji2015DocumentCL,
  title={Document Context Language Models},
  author={Yangfeng Ji and Trevor Cohn and Lingpeng Kong and Chris Dyer and Jacob Eisenstein},
  journal={ArXiv},
  year={2015},
  volume={abs/1511.03962}
}
  • Yangfeng Ji, Trevor Cohn, +2 authors Jacob Eisenstein
  • Published 2015
  • Computer Science, Mathematics
  • ArXiv
  • Text documents are structured on multiple levels of detail: individual words are related by syntax, but larger units of text are related by discourse structure. Existing language models generally fail to account for discourse structure, but it is crucial if we are to have language models that reward coherence and generate coherent texts. We present and empirically evaluate a set of multi-level recurrent neural network language models, called Document-Context Language Models (DCLM), which… CONTINUE READING
    58 Citations

    Figures, Tables, and Topics from this paper

    Topically Driven Neural Language Model
    • 36
    • PDF
    Dialog context language modeling with recurrent neural networks
    • Bing Liu, I. Lane
    • Computer Science
    • 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
    • 2017
    • 21
    • Highly Influenced
    • PDF
    Larger-Context Language Modelling with Recurrent Neural Network
    • 37
    • PDF
    Context Models for OOV Word Translation in Low-Resource Languages
    • 3
    • PDF
    A Latent Variable Recurrent Neural Network for Discourse-Driven Language Models
    • 88
    • PDF
    Document Context Neural Machine Translation with Memory Networks
    • 87
    • PDF
    A Latent Variable Recurrent Neural Network for Discourse Relation Language Models
    • 52
    Contextual LSTM (CLSTM) models for Large scale NLP tasks
    • 147
    • PDF
    A Parallel Recurrent Neural Network for Language Modeling with POS Tags
    • 4
    • Highly Influenced
    • PDF
    Training Language Models for Long-Span Cross-Sentence Evaluation
    • 11
    • PDF

    References

    SHOWING 1-10 OF 41 REFERENCES
    Larger-Context Language Modelling
    • 45
    Hierarchical Recurrent Neural Network for Document Modeling
    • 124
    • Highly Influential
    • PDF
    Context dependent recurrent neural network language model
    • Tomas Mikolov, G. Zweig
    • Computer Science
    • 2012 IEEE Spoken Language Technology Workshop (SLT)
    • 2012
    • 477
    • PDF
    A Model of Coherence Based on Distributed Sentence Representation
    • 98
    • PDF
    Half-Context Language Models
    • 5
    • PDF
    A Hierarchical Neural Autoencoder for Paragraphs and Documents
    • 477
    • Highly Influential
    • PDF
    Docent: A Document-Level Decoder for Phrase-Based Statistical Machine Translation
    • 59
    • PDF
    Distributed Representations of Sentences and Documents
    • 5,612
    • PDF
    Modeling Local Coherence: An Entity-Based Approach
    • 594
    • PDF
    Statistical Machine Translation
    • M. Osborne
    • Computer Science
    • Encyclopedia of Machine Learning and Data Mining
    • 2010
    • 809
    • PDF