Inference for the Generalization Error

@article{Nadeau1999InferenceFT,
  title={Inference for the Generalization Error},
  author={Claude Nadeau and Yoshua Bengio},
  journal={Machine Learning},
  year={1999},
  volume={52},
  pages={239-281}
}
In order to compare learning algorithms, experimental results reported in the machine learning literature often use statistical tests of significance to support the claim that a new learning algorithm generalizes better. Such tests should take into account the variability due to the choice of training set and not only that due to the test examples, as is often the case. This could lead to gross underestimation of the variance of the cross-validation estimator, and to the wrong conclusion that… CONTINUE READING
Highly Influential
This paper has highly influenced a number of papers. REVIEW HIGHLY INFLUENTIAL CITATIONS