Local Rademacher complexities

@article{Bartlett2005LocalRC,
  title={Local Rademacher complexities},
  author={Peter L. Bartlett and Olivier Bousquet and Shahar Mendelson},
  journal={Annals of Statistics},
  year={2005},
  volume={33},
  pages={1497-1537}
}
We propose new bounds on the error of learning algorithms in terms of a data-dependent notion of complexity. The estimates we establish give optimal rates and are based on a local and empirical version of Rademacher averages, in the sense that the Rademacher averages are computed from the data, on a subset of functions with small empirical error. We present some applications to classification and prediction with convex function classes, and with kernel classes in particular. 
Learning Kernels Using Local Rademacher Complexity
TLDR
Two new learning kernel algorithms are devised: one based on a convex optimization problem for which the author gives an efficient solution using existing learning kernel techniques, and another that can be formulated as a DC-programming problem forwhich the author describes a solution in detail. Expand
Local Rademacher complexity bounds based on covering numbers
TLDR
The power of the complexity bounds is demonstrated by applying them to derive effective generalization error bounds and could yield refined local Rademacher complexity bounds for function classes satisfying general entropy conditions. Expand
Rademacher complexity in Neyman-Pearson classification
Neyman-Pearson(NP) criterion is one of the most important ways in hypothesis testing. It is also a criterion for classification. This paper addresses the problem of bounding the estimation error ofExpand
Rademacher Complexities and Bounding the Excess Risk in Active Learning
TLDR
Sequential algorithms of active learning based on the estimation of the level sets of the empirical risk are discussed in the paper and several applications to binary classification problems are considered. Expand
Transductive Rademacher Complexity and Its Applications
TLDR
A new PAC-Bayesian bound for mixtures of transductive algorithms based on a new Rademacher bound based on the "unlabeled-labeled" decomposition technique applies to many current and practical graph-based algorithms. Expand
The Local Rademacher Complexity of Lp-Norm Multiple Kernel Learning
TLDR
An upper bound is derived on the local Rademacher complexity of lp-norm multiple kernel learning, which yields a tighter excess risk bound than global approaches and consequences regarding excess loss are derived. Expand
Bounds for Learning the Kernel: Rademacher Chaos Complexity
In this paper we develop a novel probabilistic generalization bound for regularized kernel learning algorithms. First, we show that generalization analysis of kernel learning algorithms reduces toExpand
Local Rademacher Complexity: Sharper risk bounds with and without unlabeled samples
TLDR
A new Local Rademacher Complexity risk bound is derived on the generalization ability of a model, which is able to take advantage of the availability of unlabeled samples, which improves state-of-the-art results even when no unlabeling samples are available. Expand
Rademacher Chaos Complexities for Learning the Kernel Problem
TLDR
A novel generalization bound for learning the kernel problem is developed and satisfactory excess generalization bounds and misclassification error rates for learning gaussian kernels and general radial basis kernels are established. Expand
Fast convergence of extended Rademacher Complexity bounds
In this work we propose some new generalization bounds for binary classifiers, based on global Rademacher Complexity (RC), which exhibit fast convergence rates by combining state-of-the-art resultsExpand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 76 REFERENCES
Rademacher and Gaussian Complexities: Risk Bounds and Structural Results
TLDR
This work investigates the use of certain data-dependent estimates of the complexity of a function class called Rademacher and Gaussian complexities and proves general risk bounds in terms of these complexities in a decision theoretic setting. Expand
Rademacher Processes and Bounding the Risk of Function Learning
We construct data dependent upper bounds on the risk in function learning problems. The bounds are based on local norms of the Rademacher process indexed by the underlying function class, and they doExpand
Some Local Measures of Complexity of Convex Hulls and Generalization Bounds
We investigate measures of complexity of function classes based on continuity moduli of Gaussian and Rademacher processes. For Gaussian processes, we obtain bounds on the continuity modulus on theExpand
Improving the sample complexity using global data
  • S. Mendelson
  • Mathematics, Computer Science
  • IEEE Trans. Inf. Theory
  • 2002
TLDR
This work improves the known estimates for classes which have relatively small covering numbers in empirical L/sub 2/ spaces (e.g. log-covering numbers which are polynomial with exponent p<2), and presents several examples of relevant classes which has a "small" fat-shattering dimension and hence fit the setup. Expand
Rademacher penalties and structural risk minimization
TLDR
This work suggests a penalty function to be used in various problems of structural risk minimization, based on the sup-norm of the so-called Rademacher process indexed by the underlying class of functions (sets), and obtains oracle inequalities for the theoretical risk of estimators, obtained by structural minimization of the empirical risk withRademacher penalties. Expand
Empirical minimization
We investigate the behavior of the empirical minimization algorithm using various methods. We first analyze it by comparing the empirical, random, structure and the original one on the class, eitherExpand
The importance of convexity in learning with squared loss
TLDR
It is shown that if the closure of a function class under the metric induced by some probability distribution is not convex, then the sample complexity for agnostically learning with squared loss is lower than that for agnostic learning, so learning the convex hull provides better approximation capabilities with little sample complexity penalty. Expand
Rademacher averages and phase transitions in Glivenko-Cantelli classes
  • S. Mendelson
  • Mathematics, Computer Science
  • IEEE Trans. Inf. Theory
  • 2002
TLDR
The best known estimates on the covering numbers of a class when considered as a subset of L/sub p/ spaces are proved and the fat-shattering dimension of the convex hull of a given class is estimated. Expand
Bounding the generalization error of convex combinations of classifiers: balancing the dimensionality and the margins
A problem of bounding the generalization error of a classifier f in H, where H is a "base" class of functions (classifiers), is considered. This problem frequently occurs in computer learning, whereExpand
Concentration
TLDR
This paper introduces and surveys two approaches for bounding probabilities of large deviations for sums of bounded independent random variables, and some of their applications. Expand
...
1
2
3
4
5
...