Corpus ID: 224803726

Non-parametric Binary regression in metric spaces with KL loss

@article{Avital2020NonparametricBR,
  title={Non-parametric Binary regression in metric spaces with KL loss},
  author={Ariel Avital and Klim Efremenko and Aryeh Kontorovich and David Toplin and Bo Waggoner},
  journal={ArXiv},
  year={2020},
  volume={abs/2010.09886}
}
We propose a non-parametric variant of binary regression, where the hypothesis is regularized to be a Lipschitz function taking a metric space to [0,1] and the loss is logarithmic. This setting presents novel computational and statistical challenges. On the computational front, we derive a novel efficient optimization algorithm based on interior point methods; an attractive feature is that it is parameter-free (i.e., does not require tuning an update step size). On the statistical front, the… Expand

References

SHOWING 1-10 OF 30 REFERENCES
Efficient Regression in Metric Spaces via Approximate Lipschitz Extension
TLDR
A regression algorithm whose speed and generalization performance depend on the intrinsic dimension of the data, to which the algorithm adapts is designed, and the main innovation is algorithmic. Expand
Efficient Classification for Metric Data
TLDR
A new algorithm for classification in general metric spaces, whose runtime and accuracy depend on the doubling dimension of the data points, and can thus achieve superior classification performance in many common scenarios. Expand
Distance-Based Classification with Lipschitz Functions
TLDR
The goal of this article is to find a generalization of linear decision functions for metric spaces and define a corresponding notion of margin such that the decision function separates the training points with a large margin. Expand
Rademacher and Gaussian Complexities: Risk Bounds and Structural Results
TLDR
This work investigates the use of certain data-dependent estimates of the complexity of a function class called Rademacher and Gaussian complexities and proves general risk bounds in terms of these complexities in a decision theoretic setting. Expand
Nonparametric binary regression using a Gaussian process prior
Abstract The article describes a nonparametric Bayesian approach to estimating the regression function for binary response data measured with multiple covariates. A multiparameter Gaussian process,Expand
Structural Risk Minimization Over Data-Dependent Hierarchies
TLDR
A result is presented that allows one to trade off errors on the training sample against improved generalization performance, and a more general result in terms of "luckiness" functions, which provides a quite general way for exploiting serendipitous simplicity in observed data to obtain better prediction accuracy from small training sets. Expand
CONVEX OPTIMIZATION, SHAPE CONSTRAINTS, COMPOUND DECISIONS, AND EMPIRICAL BAYES RULES
Estimation of mixture densities for the classical Gaussian compound decision problem and their associated (empirical) Bayes rules is considered from two new perspectives. The first, motivated byExpand
Computing Maximum Likelihood Estimators of Convex Density Functions
TLDR
This work considers the problem of estimating a density function that is known in advance to be convex, and proposes a clustering scheme that is reasonable from a statistical point of view and shows that interior point algorithms perform well on this class of optimization problem. Expand
Convex Optimization, Shape Constraints, Compound Decisions, and Empirical Bayes Rules
Estimation of mixture densities for the classical Gaussian compound decision problem and their associated (empirical) Bayes rules is considered from two new perspectives. The first, motivated byExpand
Introduction to Nonparametric Estimation
  • A. Tsybakov
  • Computer Science, Mathematics
  • Springer series in statistics
  • 2009
TLDR
The main idea is to introduce the fundamental concepts of the theory while maintaining the exposition suitable for a first approach in the field, and many important and useful results on optimal and adaptive estimation are provided. Expand
...
1
2
3
...