• Corpus ID: 851184

Beyond Disagreement-Based Agnostic Active Learning

  title={Beyond Disagreement-Based Agnostic Active Learning},
  author={Chicheng Zhang and Kamalika Chaudhuri},
We study agnostic active learning, where the goal is to learn a classifier in a pre-specified hypothesis class interactively with as few label queries as possible, while making no assumptions on the true function generating the labels. The main algorithms for this problem are {\em{disagreement-based active learning}}, which has a high label requirement, and {\em{margin-based active learning}}, which only applies to fairly restricted settings. A major challenge is to find an algorithm which… 

Improved Algorithms for Agnostic Pool-based Active Classification

This work proposes an algorithm that, in contrast to uniform sampling over the disagreement region, solves an experimental design problem to determine a distribution over examples from which to request labels and demonstrates that the algorithm is superior to state of the art agnostic active learning algorithms on image classification datasets.

Active Heteroscedastic Regression

The results illustrate that just like in binary classification, some partial knowledge of the nature of the noise can lead to significant gains in the label requirement of active learning.

Convergence Rates of Active Learning for Maximum Likelihood Estimation

This paper provides an upper bound on the label requirement of the algorithm, and a lower bound that matches it up to lower order terms, and shows that unlike binary classification in the realizable case, just a single extra round of interaction is sufficient to achieve near-optimal performance in maximum likelihood estimation.

Active Learning for Cost-Sensitive Classification

It is proved COAL can be efficiently implemented for any regression family that admits squared loss optimization; it also enjoys strong guarantees with respect to predictive performance and labeling effort.

Efficient Active Learning with Abstention

The first computationally computationally active learning algorithm with abstention is developed, guaranteed to only abstain on hard examples (where the true label distribution is close to a fair coin), a novel property the authors term “proper abstention” that also leads to a host of other desirable characteristics.

Active learning from noisy and abstention feedback

This work considers a more general setting where the labeling oracle can abstain from providing a label in addition to returning noisy labels and provides a model for this setting, and provides matching lower bounds to demonstrate that the algorithm has near-optimal estimation accuracy.

Online Active Model Selection for Pre-trained Classifiers

This work designs an online selective sampling approach that actively selects informative examples to label and outputs the best model with high probability at any round.

Revisiting Model-Agnostic Private Learning: Faster Rates and Active Learning

A new Tsybakov Noise Condition is introduced and stronger and more interpretable learning bounds are established to provide new insights into when PATE works and improve over existing results even in the narrower realizable setting.

Selectively Inhibiting Learning Bias for Active Sampling

A simple hybrid strategy and a visualization tool called ranking curves, which allowed to see clearly when the presence of a learner was possibly detrimental and was successfully compared to its counterpart in the literature, to pure agnostic strategies and to the usual baseline of the field.

Efficient and Parsimonious Agnostic Active Learning

We develop a new active learning algorithm for the streaming setting satisfying three important properties: 1) It provably works for any classifier representation and classification problem including



A General Agnostic Active Learning Algorithm

This work presents an agnostic active learning algorithm for any hypothesis class of bounded VC dimension under arbitrary data distributions, using reductions to supervised learning that harness generalization bounds in a simple but subtle manner and provides a fall-back guarantee that bounds the algorithm's label complexity by the agnostic PAC sample complexity.

Algorithms for active learning

This dissertation develops and analyzes active learning algorithms for binary classification problems based on maintaining a version space and develops algorithms based on simpler reductions to agnostic learning that more closely match the standard abstraction of supervised learning procedures.

Agnostic Selective Classification

It is theoretically possible to track the same classification performance of the best (unknown) hypothesis in their class, provided that the authors are free to abstain from prediction in some region of their choice, and a novel selective classification algorithm is developed using constrained SVMs.

Agnostic active learning

The first active learning algorithm which works in the presence of arbitrary forms of noise is state and analyzed, and it is shown that A2 achieves an exponential improvement over the usual sample complexity of supervised learning.

PLAL: Cluster-based active learning

It is shown that for several common learning paradigms, applying the PLAL procedure as a preprocessing leads to provable label complexity reductions (over any “passive” learning algorithm, under the same data assumptions).

Active Learning Literature Survey

This report provides a general introduction to active learning and a survey of the literature, including a discussion of the scenarios in which queries can be formulated, and an overview of the query strategy frameworks proposed in the literature to date.

Active Learning via Perfect Selective Classification

A reduction of active learning to selective classification that preserves fast rates is shown and exponential target-independent label complexity speedup is derived for actively learning general (non-homogeneous) linear classifiers when the data distribution is an arbitrary high dimensional mixture of Gaussians.

Agnostic Active Learning Without Constraints

We present and analyze an agnostic active learning algorithm that works without keeping a version space. This is unlike all previous approaches where a restricted set of candidate hypotheses is

A bound on the label complexity of agnostic active learning

General bounds on the number of label requests made by the A2 algorithm proposed by Balcan, Beygelzimer & Langford are derived, which represents the first nontrivial general-purpose upper bound on label complexity in the agnostic PAC model.

Improving generalization with active learning

A formalism for active concept learning called selective sampling is described and it is shown how it may be approximately implemented by a neural network.