- Full text PDF available (6)
- This year (0)
- Last 5 years (0)
- Last 10 years (0)
Journals and Conferences
We combine the replica approach from statistical physics with a varia-tional approach to analyze learning curves analytically. We apply the method to Gaussian process regression. As a main result we derive ap-proximative relations between empirical error measures, the generalization error and the posterior variance.
Based on a statistical mechanics approach, we develop a method for approximately computing average case learning curves for Gaus-sian process regression models. The approximation works well in the large sample size limit and for arbitrary dimensionality of the input space. We explain how the approximation can be systematically improved and argue that… (More)
We compute approximate analytical bootstrap averages for support vector classification using a combination of the replica method of statistical physics and the TAP approach for approximate inference. We test our method on a few datasets and compare it with exact averages obtained by extensive Monte-Carlo sampling.
Using a variational technique, we generalize the statistical physics approach of learning from random examples to make it applicable to real data. We demonstrate the validity and relevance of our method by computing approximate estimators for generalization errors that are based on training data alone.
Using a novel reformulation, we develop a framework to compute approximate resampling data averages analytically. The method avoids multiple retraining of statistical models on the samples. Our approach uses a combination of the replica " trick " of statistical physics and the TAP approach for approximate Bayesian inference. We demonstrate our approach on… (More)
We apply the replica method of Statistical Physics combined with a vari-ational method to the approximate analytical computation of bootstrap averages for estimating the generalization error. We demonstrate our approach on regression with Gaussian processes and compare our results with averages obtained by Monte-Carlo sampling.
We employ the replica method of statistical physics to study the average case performance of learning systems. The new feature of our theory is that general distributions of data can be treated, which enables applications to real data. For a class of Bayesian prediction models which are based on Gaussian processes, we discuss Bootstrap estimates for… (More)