• Corpus ID: 12487255

Sequential Nonparametric Regression

  title={Sequential Nonparametric Regression},
  author={Haijie Gu and John D. Lafferty},
We present algorithms for nonparametric regression in settings where the data are obtained sequentially. While traditional estimators select bandwidths that depend upon the sample size, for sequential data the effective sample size is dynamically changing. We propose a linear time algorithm that adjusts the bandwidth for each new data point, and show that the estimator achieves the optimal minimax rate of convergence. We also propose the use of online expert mixing algorithms to adapt to… 
Recursive Nonparametric Estimation for Time Series
This paper proves that the proposed estimators have the asymptotic normality and the corresponding central limit theorems are provided, and establishes the sharp laws of the iterated logarithms that precisely characterize the ascyptotic almost sure behavior of the proposed estimation estimators.
Regression-tree Tuning in a Streaming Setting
It is proved that it is possible to maintain a partition-based regression structure in time O (log n) at any time step n while achieving a nearly-optimal regression rate of O (n-2/(2+d)) in terms of the unknown metric dimension d.
Gradients Weights improve Regression and Classification
A simple estimator of these derivative norms is proposed and it is shown that weighting each coordinate i according to an estimate of the variation of f along coordinate i is an efficient way to significantly improve the performance of distance-based regressors such as kernel and k- NN regressors.


Sequential Data-Adaptive Bandwidth Selection by Cross-Validation for Nonparametric Prediction
We consider the problem of bandwidth selection by cross-validation from a sequential point of view in a nonparametric regression model. Having in mind that in applications one often aims at
Adaptive Regression by Mixing
Adaptation over different procedures is of practical importance. Different procedures perform well under different conditions. In many practical situations, it is rather hard to assess which
Data‐Driven Bandwidth Selection in Local Polynomial Fitting: Variable Bandwidth and Spatial Adaptation
When estimating a mean regression function and its derivatives, locally weighted least squares regression has proven to be a very attractive technique. The present paper focuses on the important
Sequential Procedures for Aggregating Arbitrary Estimators of a Conditional Mean
  • F. Bunea, A. Nobel
  • Mathematics, Computer Science
    IEEE Transactions on Information Theory
  • 2008
From the cumulative loss bounds, an oracle inequality is derived for the aggregate estimator for an unbounded response having a suitable moment-generating function that readily yields convergence rates for aggregation over the unit simplex that are within logarithmic factors of known minimax bounds.
An Effective Bandwidth Selector for Local Least Squares Regression
Abstract Local least squares kernel regression provides an appealing solution to the nonparametric regression, or “scatterplot smoothing,” problem, as demonstrated by Fan, for example. The practical
Online kernel density estimation for interactive learning
A Gaussian-kernel-based online kernel density estimation which can be used for applications of online probability density estimation and online learning and allows online adaptation from positive examples as well as from the negative examples.
Local polynomial fitting: A standard for nonparametric regression
Among the various nonparametric regression methods, weighted local polynomial fitting is the one which is gaining increasing popularity. This is due to the attractive minimax efficiency of the method
Optimal spatial adaptation to inhomogeneous smoothness: an approach based on kernel estimates with variable bandwidth selectors
A new v~ria~~,E_<:.~_~1~!~_s,~~~,!?E_0r estimation is proposed. The application-of this bandwidth selector leads to kernel estimates that achieve optimal rates of convergence over B£~~.£~:3.sses.
Generalized Additive Models
The class of generalized additive models is introduced, which replaces the linear form E fjXj by a sum of smooth functions E sj(Xj), and has the advantage of being completely auto- matic, i.e., no "detective work" is needed on the part of the statistician.
We study some methods of combining procedures for forecasting a continuous random variable. Statistical risk bounds under the square error loss are obtained under distributional assumptions on the