Adaptive Stratified Sampling for Precision-Recall Estimation

@inproceedings{Sabharwal2018AdaptiveSS,
  title={Adaptive Stratified Sampling for Precision-Recall Estimation},
  author={Ashish Sabharwal and Yexiang Xue},
  booktitle={UAI},
  year={2018}
}
We propose a new algorithm for computing a constant-factor approximation of precision-recall (PR) curves for massive noisy datasets produced by generative models. Assessing validity of items in such datasets requires human annotation, which is costly and must be minimized. Our algorithm, AdaStrat, is the first data-aware method for this task. It chooses the next point to query on the PR curve adaptively, based on previous observations. It then selects specific items to annotate using stratified… CONTINUE READING

Citations

Publications citing this paper.

References

Publications referenced by this paper.
SHOWING 1-10 OF 16 REFERENCES