# Sequential Nonparametric Regression

@article{Gu2012SequentialNR, title={Sequential Nonparametric Regression}, author={Haijie Gu and John D. Lafferty}, journal={ArXiv}, year={2012}, volume={abs/1206.6408} }

We present algorithms for nonparametric regression in settings where the data are obtained sequentially. While traditional estimators select bandwidths that depend upon the sample size, for sequential data the effective sample size is dynamically changing. We propose a linear time algorithm that adjusts the bandwidth for each new data point, and show that the estimator achieves the optimal minimax rate of convergence. We also propose the use of online expert mixing algorithms to adapt to…

## 3 Citations

Recursive Nonparametric Estimation for Time Series

- Mathematics, Computer ScienceIEEE Transactions on Information Theory
- 2014

This paper proves that the proposed estimators have the asymptotic normality and the corresponding central limit theorems are provided, and establishes the sharp laws of the iterated logarithms that precisely characterize the ascyptotic almost sure behavior of the proposed estimation estimators.

Regression-tree Tuning in a Streaming Setting

- Computer Science, MathematicsNIPS
- 2013

It is proved that it is possible to maintain a partition-based regression structure in time O (log n) at any time step n while achieving a nearly-optimal regression rate of O (n-2/(2+d)) in terms of the unknown metric dimension d.

Gradients Weights improve Regression and Classification

- Mathematics, Computer ScienceJ. Mach. Learn. Res.
- 2016

A simple estimator of these derivative norms is proposed and it is shown that weighting each coordinate i according to an estimate of the variation of f along coordinate i is an efficient way to significantly improve the performance of distance-based regressors such as kernel and k- NN regressors.

## References

SHOWING 1-10 OF 15 REFERENCES

Sequential Data-Adaptive Bandwidth Selection by Cross-Validation for Nonparametric Prediction

- Mathematics
- 2012

We consider the problem of bandwidth selection by cross-validation from a sequential point of view in a nonparametric regression model. Having in mind that in applications one often aims at…

Adaptive Regression by Mixing

- Mathematics
- 2001

Adaptation over different procedures is of practical importance. Different procedures perform well under different conditions. In many practical situations, it is rather hard to assess which…

Data‐Driven Bandwidth Selection in Local Polynomial Fitting: Variable Bandwidth and Spatial Adaptation

- Mathematics
- 1995

When estimating a mean regression function and its derivatives, locally weighted least squares regression has proven to be a very attractive technique. The present paper focuses on the important…

Sequential Procedures for Aggregating Arbitrary Estimators of a Conditional Mean

- Mathematics, Computer ScienceIEEE Transactions on Information Theory
- 2008

From the cumulative loss bounds, an oracle inequality is derived for the aggregate estimator for an unbounded response having a suitable moment-generating function that readily yields convergence rates for aggregation over the unit simplex that are within logarithmic factors of known minimax bounds.

An Effective Bandwidth Selector for Local Least Squares Regression

- Mathematics
- 1995

Abstract Local least squares kernel regression provides an appealing solution to the nonparametric regression, or “scatterplot smoothing,” problem, as demonstrated by Fan, for example. The practical…

Online kernel density estimation for interactive learning

- Mathematics, Computer ScienceImage Vis. Comput.
- 2010

A Gaussian-kernel-based online kernel density estimation which can be used for applications of online probability density estimation and online learning and allows online adaptation from positive examples as well as from the negative examples.

Local polynomial fitting: A standard for nonparametric regression

- Mathematics
- 1993

Among the various nonparametric regression methods, weighted local polynomial fitting is the one which is gaining increasing popularity. This is due to the attractive minimax efficiency of the method…

Optimal spatial adaptation to inhomogeneous smoothness: an approach based on kernel estimates with variable bandwidth selectors

- Mathematics
- 1997

A new v~ria~~,E_<:.~_~1~!~_s,~~~,!?E_0r estimation is proposed. The application-of this bandwidth selector leads to kernel estimates that achieve optimal rates of convergence over B£~~.£~:3.sses.…

Generalized Additive Models

- Computer Science, Mathematics
- 1990

The class of generalized additive models is introduced, which replaces the linear form E fjXj by a sum of smooth functions E sj(Xj), and has the advantage of being completely auto- matic, i.e., no "detective work" is needed on the part of the statistician.

COMBINING FORECASTING PROCEDURES: SOME THEORETICAL RESULTS

- MathematicsEconometric Theory
- 2004

We study some methods of combining procedures for forecasting a continuous random variable. Statistical risk bounds under the square error loss are obtained under distributional assumptions on the…