Statistical Theory of Overtraining - Is Cross-Validation Asymptotically Effective?

@inproceedings{Amari1995StatisticalTO,
  title={Statistical Theory of Overtraining - Is Cross-Validation Asymptotically Effective?},
  author={Shun-ichi Amari and Noboru Murata and Klaus-Robert M{\"u}ller and Michael Finke and Howard Hua Yang},
  booktitle={NIPS},
  year={1995}
}
A statistical theory for overtraining is proposed. The analysis treats realizable stochastic neural networks, trained with KullbackLeibler loss in the asymptotic case. It is shown that the asymptotic gain in the generalization error is small if we perform early stopping, even if we have access to the optimal stopping time. Considering cross-validation… CONTINUE READING