Extending a Parser to Distant Domains Using a Few Dozen Partially Annotated Examples

@inproceedings{Joshi2018ExtendingAP,
  title={Extending a Parser to Distant Domains Using a Few Dozen Partially Annotated Examples},
  author={Vidur Joshi and Matthew E. Peters and Mark Hopkins},
  booktitle={ACL},
  year={2018}
}
We revisit domain adaptation for parsers in the neural era. First we show that recent advances in word representations greatly diminish the need for domain adaptation when the target domain is syntactically similar to the source domain. As evidence, we train a parser on the Wall Street Journal alone that achieves over 90% F1 on the Brown corpus. For more syntactically distant domains, we provide a simple way to adapt a parser using only dozens of partial annotations. For instance, we increase… Expand
Cross-Domain Generalization of Neural Constituency Parsers
TLDR
It is shown that despite the rich input representations they learn, neural parsers still benefit from structured output prediction of output trees, yielding higher exact match accuracy and stronger generalization both to larger text spans and to out-of-domain corpora. Expand
Semi-supervised Domain Adaptation for Dependency Parsing
TLDR
A simple domain embedding approach is proposed to merge the source- and target-domain training data, which is shown to be more effective than both direct corpus concatenation and multi-task learning and shows that a simple fine-tuning procedure can further boost cross-domain parsing accuracy by large margin. Expand
Deep Contextualized Word Embeddings for Universal Dependency Parsing
TLDR
Based on ELMo’s advantage on OOV, experiments that simulate low-resource settings are conducted and the results show that deep contextualized word embeddings are effective for data-insufficient tasks where the OOV problem is severe. Expand
Bidirectional Domain Adaptation Using Weighted Multi-Task Learning
Domain adaption in syntactic parsing is still a significant challenge. We address the issue of data imbalance between the in-domain and out-of-domain treebank typically used for the problem. WeExpand
Parsing Early Modern English for Linguistic Search
TLDR
This work trains a part-of-speech tagger and parser on a corpus of historical English, using ELMo embeddings trained over a billion words of similar text, to investigate the question of whether advances in NLP make it possible to vastly increase the size of data usable for research in historical syntax. Expand
Dissecting Contextual Word Embeddings: Architecture and Representation
TLDR
There is a tradeoff between speed and accuracy, but all architectures learn high quality contextual representations that outperform word embeddings for four challenging NLP tasks, suggesting that unsupervised biLMs, independent of architecture, are learning much more about the structure of language than previously appreciated. Expand
Multilingual Constituency Parsing with Self-Attention and Pre-Training
TLDR
It is shown that constituency parsing benefits from unsupervised pre-training across a variety of languages and a range of pre- training conditions, and the idea of joint fine-tuning is explored and shows that it gives low-resource languages a way to benefit from the larger datasets of other languages. Expand
Transductive Learning of Neural Language Models for Syntactic and Semantic Analysis
TLDR
This work fine-tune language models (LMs) on an unlabeled test set to obtain test-set-specific word representations and demonstrates that despite its simplicity, transductive LM fine-tuning consistently improves state- of-the-art neural models in in-domain and out-of-domain settings. Expand
Improving Compositional Generalization in Semantic Parsing
TLDR
This work analyzes a wide variety of models and proposes multiple extensions to the attention module of the semantic parser, aiming to improve compositional generalization in semantic parsing, as output programs are constructed from sub-components. Expand
Improving Constituency Parsing with Span Attention
TLDR
This paper proposes span attention for neural chart-based constituency parsing to leverage n-gram information, and proposes categorical span attention to further enhance the model by weighting n-rams within different length categories, and thus benefit long-sentence parsing. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 48 REFERENCES
Automatic Domain Adaptation for Parsing
TLDR
The resulting system proposes linear combinations of parsing models trained on the source corpora that outperforms all non-oracle baselines including the best domain-independent parsing model. Expand
Training Parsers on Partial Trees: A Cross-language Comparison
TLDR
This study compares data-driven dependency parsers obtained by means of annotation projection between language pairs of varying structural similarity and finds that the projected parsers substantially outperform the authors' heuristic baselines by 9―25% UAS, which corresponds to a 21―43% reduction in error rate. Expand
Combining Active Learning and Partial Annotation for Domain Adaptation of a Japanese Dependency Parser
TLDR
This paper shows that active learning can be used for domain adaptation of dependency parsers, not just in single-domain settings, and it is found that partial annotation delivers better indomain performance for the same amount of human effort than full annotation. Expand
Any domain parsing: automatic domain adaptation for natural language parsing
TLDR
A technique is presented, Any Domain Parsing, which automatically detects useful source domains and mixes them together to produce a customized parsing model which performs almost as well as the best seen parsing models (oracle) for each target domain. Expand
Self-Training for Enhancement and Domain Adaptation of Statistical Parsers Trained on Small Datasets
TLDR
This paper uses selftraining in order to improve the quality of a parser and to adapt it to a different domain, using only small amounts of manually annotated seed data, for the first time that self-training with small labeled datasets is applied successfully to these tasks. Expand
Reranking and Self-Training for Parser Adaptation
TLDR
The reranking parser described in Charniak and Johnson (2005) improves performance of the parser on Brown to 85.2% and use of the self-training techniques described in (McClosky et al., 2006) raise this to 87.8% (an error reduction of 28%) again without any use of labeled Brown data. Expand
Domain Adaptation for Dependency Parsing via Self-Training
TLDR
This paper improves parsing accuracy for out-of-domain texts with a self-training approach that uses confidence-based methods to select additional training samples that is substantially higher than improvements reported in previous work. Expand
A Maximum-Entropy-Inspired Parser
TLDR
A new parser for parsing down to Penn tree-bank style parse trees that achieves 90.1% average precision/recall for sentences of length 40 and less and 89.5% when trained and tested on the previously established sections of the Wall Street Journal treebank is presented. Expand
Training Dependency Parsers with Partial Annotation
TLDR
This paper describes and compares two straightforward approaches for three mainstream dependency parsers and proposes the second approach to directly training a linear graph-based parse (LGPar) and a linear transition-based parser (LTPar) with PA based on the idea of constrained decoding. Expand
Dependency Parsing and Domain Adaptation with LR Models and Parser Ensembles
TLDR
A data-driven variant of the LR algorithm for dependency parsing is presented, and extended with a best-first search for probabilistic generalized LR dependency parsing, and applied to both tracks of the CoNLL 2007 shared task. Expand
...
1
2
3
4
5
...