Semi-Supervised Sequence Modeling with Cross-View Training

@inproceedings{Clark2018SemiSupervisedSM,
  title={Semi-Supervised Sequence Modeling with Cross-View Training},
  author={Kevin Clark and Minh-Thang Luong and Christopher D. Manning and Quoc V. Le},
  booktitle={EMNLP},
  year={2018}
}
  • Kevin Clark, Minh-Thang Luong, +1 author Quoc V. Le
  • Published in EMNLP 2018
  • Computer Science
  • Unsupervised representation learning algorithms such as word2vec and ELMo improve the accuracy of many supervised NLP models, mainly because they can take advantage of large amounts of unlabeled text. [...] Key Method On labeled examples, standard supervised learning is used. On unlabeled examples, CVT teaches auxiliary prediction modules that see restricted views of the input (e.g., only part of a sentence) to match the predictions of the full model seeing the whole input. Since the auxiliary modules and the…Expand Abstract
    BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
    • 10,542
    • Highly Influenced
    • PDF
    Semi-Supervised Semantic Role Labeling with Cross-View Training
    • 6
    • Highly Influenced
    • PDF
    Revisiting Self-Training for Neural Sequence Generation
    • 32
    • PDF
    Semi-Supervised Learning for Text Classification by Layer Partitioning
    • 1
    • PDF
    Unsupervised Data Augmentation for Consistency Training
    • 174
    • PDF
    Semi-supervised Thai Sentence Segmentation Using Local and Distant Word Representations
    • 1
    • Highly Influenced
    • PDF
    SC-LSTM: Learning Task-Specific Representations in Multi-Task Learning for Sequence Labeling
    • 4
    • Highly Influenced
    • PDF
    Unsupervised Data Augmentation
    • 115
    Semi-supervised Domain Adaptation for Dependency Parsing
    • 5
    • PDF

    References

    Publications referenced by this paper.
    SHOWING 1-10 OF 92 REFERENCES
    Semi-supervised sequence tagging with bidirectional language models
    • 329
    • PDF
    Sequence to Sequence Learning with Neural Networks
    • 10,667
    • PDF
    Semi-supervised Sequence Learning
    • 632
    • PDF
    Improving Language Understanding by Generative Pre-Training
    • 1,437
    • PDF
    Unsupervised Pretraining for Sequence to Sequence Learning
    • 185
    • PDF