Corpus ID: 232168415

Active Testing: Sample-Efficient Model Evaluation

@inproceedings{Kossen2021ActiveTS,
  title={Active Testing: Sample-Efficient Model Evaluation},
  author={Jannik Kossen and Sebastian Farquhar and Y. Gal and Tom Rainforth},
  booktitle={ICML},
  year={2021}
}
We introduce a new framework for sampleefficient model evaluation that we call active testing. While approaches like active learning reduce the number of labels needed for model training, existing literature largely ignores the cost of labeling test data, typically unrealistically assuming large test sets for model evaluation. This creates a disconnect to real applications, where test labels are important and just as expensive, e.g. for optimizing hyperparameters. Active testing addresses this… Expand

References

SHOWING 1-10 OF 56 REFERENCES
Scikit-learn: Machine Learning in Python
Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
Deep Residual Learning for Image Recognition
Active Risk Estimation
Practical Obstacles to Deploying Active Learning
Improved Regularization of Convolutional Neural Networks with Cutout
Wide Residual Networks
Bayesian Active Learning for Classification and Preference Learning
On Statistical Bias In Active Learning: How and When To Fix It
Radial Bayesian Neural Networks: Beyond Discrete Support In Large-Scale Bayesian Deep Learning
...
1
2
3
4
5
...