# A comparative study of ordinary cross-validation, v-fold cross-validation and the repeated learning-testing methods

@article{Burman1989ACS, title={A comparative study of ordinary cross-validation, v-fold cross-validation and the repeated learning-testing methods}, author={Prabir Burman}, journal={Biometrika}, year={1989}, volume={76}, pages={503-514} }

SUMMARY Concepts of v-fold cross-validation and repeated learning-testing methods have been introduced here. In many problems, these methods are computationally much less expensive than ordinary cross-validation and can be used in its place. A comparative study of these three methods has been carried out in detail.

## Tables from this paper

## 571 Citations

Some Issues in Cross-Validation

- Computer Science
- 1991

A new type of cross- validation is proposed here for model selection problems when the data is generated by a stationary process, which is an off-shoot of both ordinary cross-validation and v-fold cross- validation.

Cross-validation as the objective function for variable-selection techniques

- Computer Science
- 2003

An assessment of ten-fold and Monte Carlo cross validations for time series forecasting

- Computer Science2013 10th International Conference on Electrical Engineering, Computing Science and Automatic Control (CCE)
- 2013

Experimental results, using time series of the NN3 tournament, found that Monte Carlo cross validation is more stable than ten-fold cross validation for selecting the best forecasting model.

Multiple predicting K-fold cross-validation for model selection

- Computer Science
- 2018

This study proposes a new CV method that uses folds of the data for model validation, while the other fold is for model construction, and provides predicted values for each observation to reduce variation in the assessment due to the averaging.

On the Use of K-Fold Cross-Validation to Choose Cutoff Values and Assess the Performance of Predictive Models in Stepwise Regression

- Computer Science
- 2009

This paper addresses a methodological technique of leave-many-out cross-validation for choosing cutoff values in stepwise regression methods for simplifying the final regression model, and proposes a resampling procedure by introducing alternative estimates of boostedCross-validated PRESS values for deciding the number of observations to be omitted and number of folds/subsets subsequently in K-fold cross- validation.

Estimation of prediction error by using K-fold cross-validation

- Computer Science, BiologyStat. Comput.
- 2011

This paper investigates two families that connect the training error and K-fold cross-validation, which has a downward bias and has an upward bias.

Consistent Cross Validation with stable learners

- Computer Science
- 2022

A debiased version of the K-fold is proposed which is consistent for any uniformly stable learner and applies to the problem of model selection and demonstrates empirically the usefulness of the promoted approach on real world datasets.

A survey of cross-validation procedures for model selection

- Medicine
- 2010

This survey intends to relate the model selection performances of cross-validation procedures to the most recent advances of model selection theory, with a particular emphasis on distinguishing empirical statements from rigorous theoretical results.

On optimal data split for generalization estimation and model selection

- Computer ScienceNeural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468)
- 1999

The theoretical basics of various cross-validation techniques are described with the purpose of reliably estimating the generalization error and optimizing the model structure for reliably estimating a single location parameter.

An empirical comparison of $$V$$V-fold penalisation and cross-validation for model selection in distribution-free regression

- Computer SciencePattern Analysis and Applications
- 2014

Cases in which VFCV and $$V$$V-fold penalisation provide poor estimates of the risk, respectively, are highlighted, and a modified penalisation technique is introduced to reduce the estimation error.

## References

SHOWING 1-10 OF 18 REFERENCES

An alternative method of cross-validation for the smoothing of density estimates

- Computer Science
- 1984

An alternative method of cross-validation, based on integrated squared error, recently also proposed by Rudemo (1982), is derived, and Hall (1983) has established the consistency and asymptotic optimality of the new method.

The Predictive Sample Reuse Method with Applications

- Computer Science, Physics
- 1975

A recently devised method of prediction based on sample reuse techniques that is most useful in low structure data paradigms that involve minimal assumptions is presented.

Classification and Regression Trees

- Computer Science
- 1983

This chapter discusses tree classification in the context of medicine, where right Sized Trees and Honest Estimates are considered and Bayes Rules and Partitions are used as guides to optimal pruning.

Generalized $L-, M-$, and $R$-Statistics

- Mathematics
- 1984

Abstract : A class of statisticss generalizing U-statistics and L-statistics, and containing other varieties of statistics as well, such as trimmed U-statistics, is studied. Using the differentiable…

Estimating Optimal Transformations for Multiple Regression and Correlation.

- Mathematics
- 1985

Abstract In regression analysis the response variable Y and the predictor variables X 1 …, Xp are often replaced by functions θ(Y) and O1(X 1), …, O p (Xp ). We discuss a procedure for estimating…

Optimal Bandwidth Selection in Nonparametric Regression Function Estimation

- Mathematics
- 1985

On considere des estimateurs du noyau d'une fonction de regression multivariable et une regle de selection selon la largeur de bande formulee en terme de validation croisee

Jackknife Approximations to Bootstrap Estimates

- Mathematics
- 1984

Let T be an estimate of the form Tn = T(F ) where F is the nn n' n sample cdf of n iid observations and T is a locally quadratic functional defined on cdf's. Then, the normalized jackknife estimates…

Approximation Theorems of Mathematical Statistics

- Mathematics
- 1980

Preliminary Tools and Foundations. The Basic Sample Statistics. Transformations of Given Statistics. Asymptotic Theory in Parametric Inference. U--Statistics. Von Mises Differentiable Statistical…

Estimation of optimal transformations using D-fold cross validation and repeated learning-testing methods

- Sankhya A 51. To appear. GEISSER,
- 1989

Estimation of optimal transformations using D-fold cross validation and repeated learning-testing methods. Sankhya A 51

- Estimation of optimal transformations using D-fold cross validation and repeated learning-testing methods. Sankhya A 51
- 1989