Nonparametric Shape-Restricted Regression

@article{Guntuboyina2017NonparametricSR,
  title={Nonparametric Shape-Restricted Regression},
  author={Adityanand Guntuboyina and Bodhisattva Sen},
  journal={Statistical Science},
  year={2017}
}
We consider the problem of nonparametric regression under shape constraints. The main examples include isotonic regression (with respect to any partial order), unimodal/convex regression, additive shape-restricted regression, and constrained single index model. We review some of the theoretical properties of the least squares estimator (LSE) in these problems, emphasizing on the adaptive nature of the LSE. In particular, we study the risk behavior of the LSE, and its pointwise limiting… 
View PDF on arXiv
60 Citations
Highly Influential Citations
4
Background Citations
32
Methods Citations
8
Results Citations
3

Figures from this paper

60 Citations

Robustness of shape-restricted regression estimators: an envelope perspective

Classical least squares estimators are well-known to be robust with respect to moment assumptions concerning the error distribution in a wide variety of finite-dimensional statistical problems;…

Adaptive confidence sets in shape restricted regression

We construct adaptive confidence sets in isotonic and convex regression. In univariate isotonic regression, if the true parameter is piecewise constant with $k$ pieces, then the Least-Squares…

Editorial: Special Issue on “Nonparametric Inference Under Shape Constraints”

Shape-constrained inference usually refers to nonparametric function estimation and uncertainty quantification under qualitative shape restrictions such as monotonicity, convexity, log-concavity and…

Stratified incomplete local simplex tests for curvature of nonparametric multiple regression

Principled nonparametric tests for regression curvature in $\mathbb{R}^{d}$ are often statistically and computationally challenging. This paper introduces the stratified incomplete local simplex…

The bias of isotonic regression.

It is proved that the bias of the isotonic regression estimator scales as O(n -β/3) up to log factors, where 1 ≤ β ≤ 2 is the exponent corresponding to Hölder smoothness of the underlying mean.

Multivariate extensions of isotonic regression and total variation denoising via entire monotonicity and Hardy–Krause variation

It is shown that the risk of the entirely monotonic LSE is almost parametric (at most $1/n$ up to logarithmic factors) when the true function is well-approximable by a rectangular piecewise constant entirely Monotone function with not too many constant pieces.

Generalized nearly isotonic regression

An efficient algorithm based on the modified PAVA is developed, which utilizes the duality between the natural and expectation parameters, and a method for selecting the regularization parameter by using an information criterion is provided.

Isotonic Regression Discontinuity Designs

In isotonic regression discontinuity designs, the average outcome and the treatment assignment probability are monotone in the running variable. We introduce novel nonparametric estimators for sharp…

Nonparametric, tuning‐free estimation of S‐shaped functions

This work considers the nonparametric estimation of an S‐shaped regression function with least squares estimator, and reveals that the estimator achieves the minimax optimal rate of convergence for both the estimation of the regression function and its inflection point.

Isotonic distributional regression

Isotonic distributional regression (IDR) is a powerful non‐parametric technique for the estimation of conditional distributions under order restrictions. In a nutshell, IDR learns conditional…
...

152 References

Inference In Exponential Family Regression Models Under Certain Shape Constraints Using Inversion Based Techniques

We address the problem of pointwise estimation of a regression function under certain shape constraints, using a number of different statistics that can be viewed as measures of discrepancy from a…

Adaptive risk bounds in unimodal regression

We study the statistical properties of the least squares estimator in unimodal sequence estimation. Although closely related to isotonic regression, unimodal regression has not been as extensively…

INFERENCE USING SHAPE-RESTRICTED REGRESSION SPLINES

Regression splines are smooth, flexible, and parsimonious nonparametric function estimators. They are known to be sensitive to knot number and placement, but if assumptions such as monotonicity or…

Generalized additive and index models with shape constraints

We study generalized additive models, with shape restrictions (e.g. monotonicity, convexity and concavity) imposed on each component of the additive prediction function. We show that this framework…

ON THE DEGREES OF FREEDOM IN SHAPE-RESTRICTED REGRESSION

For the problem of estimating a regression function, μ say, subject to shape constraints, like monotonicity or convexity, it is argued that the divergence of the maximum likelihood estimator provides…

Nonparametric Least Squares Estimation of a Multivariate Convex Regression Function

This paper deals with the consistency of the least squares estimator of a convex regression function when the predictor is multidimensional. We characterize and discuss the computation of such an…

Contraction and uniform convergence of isotonic regression

We consider the problem of isotonic regression, where the underlying signal $x$ is assumed to satisfy a monotonicity constraint, that is, $x$ lies in the cone $\{ x\in\mathbb{R}^n : x_1 \leq \dots…

A Computational Framework for Multivariate Convex Regression and Its Variants

A scalable algorithmic framework based on the augmented Lagrangian method to compute the nonparametric least squares estimator (LSE) of a multivariate convex regression function is proposed and a novel approach to obtain smooth convex approximations to the fitted (piecewise affine) convex LSE is developed.

Nonparametric Estimation under Shape Constraints

This book treats the latest developments in the theory of order-restricted inference, with special attention to nonparametric methods and algorithmic aspects, which are used in computing maximum likelihood estimators and developing distribution theory for inverse problems of this type.

A general asymptotic scheme for inference under order restrictions

Limit distributions for the greatest convex minorant and its derivative are considered for a general class of stochastic processes including partial sum processes and empirical processes, for…
...