Semi-supervised sequence tagging with bidirectional language models

@inproceedings{Peters2017SemisupervisedST,
  title={Semi-supervised sequence tagging with bidirectional language models},
  author={Matthew E. Peters and Waleed Ammar and Chandra Bhagavatula and Russell Power},
  booktitle={ACL},
  year={2017}
}
Pre-trained word embeddings learned from unlabeled text have become a standard component of neural network architectures for NLP tasks. However, in most cases, the recurrent network that operates on word-level representations to produce context sensitive representations is trained on relatively little labeled data. In this paper, we demonstrate a general semi-supervised approach for adding pre- trained context embeddings from bidirectional language models to NLP systems and apply it to sequence… CONTINUE READING

Figures, Tables, Results, and Topics from this paper.

Key Quantitative Results

  • When we include the LM embeddings in our system overall performance increases from 90.87% to 91.93% F1 for the CoNLL 2003 NER task, a more then 1% absolute F1 increase, and a substantial improvement over the previous state of the art.

Citations

Publications citing this paper.
SHOWING 1-10 OF 145 CITATIONS

Contextual String Embeddings for Sequence Labeling

VIEW 6 EXCERPTS
CITES METHODS & BACKGROUND
HIGHLY INFLUENCED

Efficient Contextualized Representation: Language Model Pruning for Sequence Labeling

  • EMNLP
  • 2018
VIEW 8 EXCERPTS
CITES METHODS & BACKGROUND
HIGHLY INFLUENCED

Deep contextualized word representations

VIEW 12 EXCERPTS
CITES METHODS & BACKGROUND

Design Challenges and Misconceptions in Neural Sequence Labeling

VIEW 8 EXCERPTS
CITES BACKGROUND, METHODS & RESULTS
HIGHLY INFLUENCED

FILTER CITATIONS BY YEAR

2017
2019

CITATION STATISTICS

  • 20 Highly Influenced Citations

  • Averaged 48 Citations per year from 2017 through 2019

References

Publications referenced by this paper.
SHOWING 1-10 OF 45 REFERENCES

Named Entity Recognition with Bidirectional LSTM-CNNs

  • Transactions of the Association for Computational Linguistics
  • 2015
VIEW 12 EXCERPTS
HIGHLY INFLUENTIAL

Natural Language Processing (almost) from Scratch

  • J. Mach. Learn. Res.
  • 2011
VIEW 5 EXCERPTS
HIGHLY INFLUENTIAL

Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependencies

  • Transactions of the Association for Computational Linguistics
  • 2016
VIEW 3 EXCERPTS
HIGHLY INFLUENTIAL