• Corpus ID: 208139253

Overcoming Practical Issues of Deep Active Learning and its Applications on Named Entity Recognition

@article{Chang2019OvercomingPI,
  title={Overcoming Practical Issues of Deep Active Learning and its Applications on Named Entity Recognition},
  author={Haw-Shiuan Chang and Shankar Vembu and Sunil Mohan and Rheeya Uppaal and Andrew McCallum},
  journal={ArXiv},
  year={2019},
  volume={abs/1911.07335}
}
Existing deep active learning algorithms achieve impressive sampling efficiency on natural language processing tasks. However, they exhibit several weaknesses in practice, including (a) inability to use uncertainty sampling with black-box models, (b) lack of robustness to noise in labeling, (c) lack of transparency. In response, we propose a transparent batch active sampling framework by estimating the error decay curves of multiple feature-defined subsets of the data. Experiments on four named… 

A Survey of Deep Active Learning

A formal classification method for the existing work in deep active learning is provided, along with a comprehensive and systematic overview, to investigate whether AL can be used to reduce the cost of sample annotation while retaining the powerful learning capabilities of DL.

Contextual Multi-View Active Learning for Short Text Classification in User-Generated Data

A novel multi-view active learning model, called Context-aware Co-testing with Bagging (COCOBA), to address issues in the classification tasks tailored for a query word– e.g., detecting illness reports given the disease name.

Learning a Cost-Effective Annotation Policy for Question Answering

This paper proposes a novel framework for annotating QA datasets that entails learning a cost-effective annotation policy and a semi-supervised annotation scheme and finds that it can reduce up to 21.1% of the annotation cost.

Contextual Multi-View Query Learning for Short Text Classification in User-Generated Data

A novel multi-view active learning model, called Context-aware Co-testing with Bagging (COCOBA), to address issues in the classification tasks tailored for a query word–e.g., detecting illness reports given the disease name.

References

SHOWING 1-10 OF 51 REFERENCES

Deep Active Learning for Named Entity Recognition

By combining deep learning with active learning, the authors can outperform classical methods even with a significantly smaller amount of training data, and this work shows otherwise.

Fast and Accurate Entity Recognition with Iterated Dilated Convolutions

Iterated Dilated Convolutional Neural Networks (ID-CNNs), which have better capacity than traditional CNNs for large context and structured prediction, are proposed, which are more accurate than Bi-LSTM-CRFs while attaining 8x faster test time speeds.

Active Learning for Black-Box Semantic Role Labeling with Neural Factors

This paper proposes a neural query strategy model that embeds both language and semantic information to automatically learn the query strategy from predictions of an SRL model alone, and demonstrates the effectiveness of both this new active learning framework and the neuralquery strategy model.

Deep Bayesian Active Learning for Natural Language Processing: Results of a Large-Scale Empirical Study

A large-scale empirical study of deep active learning, addressing multiple tasks and, for each, multiple datasets, multiple models, and a full suite of acquisition functions, finds that across all settings, Bayesian active learning by disagreement significantly improves over i.i.d. baselines and usually outperforms classic uncertainty sampling.

An Analysis of Active Learning Strategies for Sequence Labeling Tasks

This paper surveys previously used query selection strategies for sequence models, and proposes several novel algorithms to address their shortcomings, and conducts a large-scale empirical comparison.

Stream-based Active Learning in the Presence of Label Noise

This work proposes an efficient method to identify and mitigate mislabelling errors for active learning in the streaming setting and derives a measure of informativeness that expresses how much the label of an instance needs to be corrected by an expert labeller.

Incremental Relabeling for Active Learning with Noisy Crowdsourced Annotations

  • Liyue ZhaoG. SukthankarR. Sukthankar
  • Computer Science
    2011 IEEE Third Int'l Conference on Privacy, Security, Risk and Trust and 2011 IEEE Third Int'l Conference on Social Computing
  • 2011
This work proposes an active learning method that is specifically designed to be robust to label noise and presents an application of the technique in the domain of activity recognition for eldercare and validate the proposed approach using both simulated and real-world experiments using Amazon Mechanical Turk.

Scalable Active Learning by Approximated Error Reduction

H hierarchical anchor graphs are utilized to construct the candidate set as well as the nearby datapoint sets of these candidates, which enables a hierarchical expansion of candidates with the increase of labels, and allows us to further accelerate the AER estimation.

Practical Obstacles to Deploying Active Learning

It is shown that while AL may provide benefits when used with specific models and for particular domains, the benefits of current approaches do not generalize reliably across models and tasks.

Reducing Labeling Effort for Structured Prediction Tasks

A new active learning paradigm is proposed which reduces not only how many instances the annotator must label, but also how difficult each instance is to annotate, which can vary widely in structured prediction tasks.
...