Corpus ID: 2191263

On-the-Job Learning with Bayesian Decision Theory

@inproceedings{Werling2015OntheJobLW,
  title={On-the-Job Learning with Bayesian Decision Theory},
  author={Keenon Werling and A. Chaganty and Percy Liang and Christopher D. Manning},
  booktitle={NIPS},
  year={2015}
}
Our goal is to deploy a high-accuracy system starting with zero training examples. We consider an on-the-job setting, where as inputs arrive, we use real-time crowd-sourcing to resolve uncertainty where needed and output our prediction when confident. As the model improves over time, the reliance on crowdsourcing queries decreases. We cast our setting as a stochastic game based on Bayesian decision theory, which allows us to balance latency, cost, and accuracy objectives in a principled way… Expand
19 Citations
A Correlated Worker Model for Grouped, Imbalanced and Multitask Data
  • 5
  • PDF
Re-Active Learning: Active Learning with Relabeling
  • 42
  • Highly Influenced
  • PDF
Visual Supervision in Bootstrapped Information Extraction
  • 4
  • PDF
A Coverage-Based Utility Model for Identifying Unknown Unknowns
  • 26
  • PDF
Empirical Methodology for Crowdsourcing Ground Truth
  • 11
  • PDF
Human-in-the-Loop Parsing
  • 21
  • PDF
...
1
2
...

References

SHOWING 1-10 OF 33 REFERENCES
Decision-Theoretic Control of Crowd-Sourced Workflows
  • 156
  • PDF
Anytime Induction of Cost-sensitive Trees
  • 11
  • PDF
Near-Optimal Bayesian Active Learning with Noisy Observations
  • 165
  • PDF
Unbiased online active learning in data streams
  • 103
  • PDF
Active Learning from Crowds
  • 284
  • PDF
Active Classification based on Value of Classifier
  • 97
  • PDF
Some label efficient learning results
  • 43
  • PDF
Large-Scale Live Active Learning: Training Object Detectors with Crawled Data and Crowds
  • 161
  • PDF
Flock: Hybrid Crowd-Machine Learning Classifiers
  • 95
  • PDF
...
1
2
3
4
...