Cross-Validation and the Estimation of Conditional Probability Densities

  title={Cross-Validation and the Estimation of Conditional Probability Densities},
  author={Peter Hall and Jeffrey S. Racine and Qi Li},
  journal={Journal of the American Statistical Association},
  pages={1015 - 1026}
  • P. Hall, J. Racine, Qi Li
  • Published 1 December 2004
  • Computer Science
  • Journal of the American Statistical Association
Many practical problems, especially some connected with forecasting, require nonparametric estimation of conditional densities from mixed data. For example, given an explanatory data vector X for a prospective customer, with components that could include the customer's salary, occupation, age, sex, marital status, and address, a company might wish to estimate the density of the expenditure, Y, that could be made by that person, basing the inference on observations of (X, Y) for previous clients… 

Adaptive Bayesian density regression for high-dimensional data

It is shown that the posterior distribution of the conditional density contracts adaptively at the truth nearly at the optimal oracle rate, determined by the unknown sparsity and smoothness levels, even in the ultra highdimensional settings where p increases exponentially with n.

Non-Parametric Estimation of Conditional Densities: A New Method

Let X = (X1,...,Xp) be a stochastic vector having joint density function fX(x) with partitions X1 = (X1,...,Xk) and X2 = (Xk+1,...,Xp). A new method for estimating the conditional density function of


Regression problems are traditionally analyzed via univariate characteristics like the regression function, scale function and marginal density of regression errors. These characteristics are useful

Converting High-Dimensional Regression to High-Dimensional Conditional Density Estimation

FlexCode is proposed, a fully nonparametric approach to conditional density estimation that reformulates CDE as a non-parametric orthogonal series problem where the expansion coefficients are estimated by regression.

Kernel Estimation of Multivariate Conditional Distributions

We consider the problem of estimating conditional probability distributions that are multivariate in both the conditioned and conditioning variable sets. This is an extension of Hall, Racine, and Li

Approximation of conditional densities by smooth mixtures of regressions

This paper shows that large nonparametric classes of conditional multivariate densities can be approximated in the Kullback‐Leibler distance by dierent specifications of finite mixtures of normal


We consider a nonparametric Bayesian model for conditional densities. The model is a finite mixture of normal distributions with covariate dependent multinomial logit mixing probabilities. A prior

Approximation of conditional densities by smooth mixtures of regressions ∗

This paper shows that large nonparametric classes of conditional multivariate densities can be approximated in the Kullback–Leibler distance by different specifications of finite mixtures of normal

Bayesian Mixture Modeling for Multivariate Conditional Distributions

We present a Bayesian mixture model for estimating the joint distribution of mixed ordinal, nominal, and continuous data conditional on a set of fixed variables. The modeling strategy is motivated by

Contribution to the Nonparametric Estimation of the Density of the Regression Errors (Doctoral Thesis)

This thesis deals with the nonparametric estimation of density f of the regression error term E of the model Y=m(X)+E, assuming its independence with the covariate X. The difficulty linked to this



A data-driven method for estimating conditional densities

This article extends the idea of cross-validation (CV) for choosing the smoothing parameter of the “double-kernel” local linear regression for estimating a conditional density and optimizes the estimated conditional density function by minimizing the integrated square error (ISE).


An estimator for a new model that nests single-index, additive, and multiplicative models is described and the new model achieves dimension reduction without the need for choosing between single- index, multiplicative, and multiplier specifications.

A crossvalidation method for estimating conditional densities

We extend the idea of crossvalidation to choose the smoothing parameters of the 'double-kernel' local linear regression for estimating a conditional density. Our selection rule optimises the

Smoothing sparse contingency tables

Smoothing has become synonymous with a variety of nonparametric methods used in the estimation of functions. To smooth is to sand away the rough edges from a set of data. More precisely, the aim of

The Statistical Analysis of Failure Time Data

This book complements the other references well, and merits a place on the bookshelf of anyone concerned with the analysis of lifetime data from any Ž eld.

Efficient Estimation of Average Treatment Effects With Mixed Categorical and Continuous Data ∗

In this paper we consider the nonparametric estimation of average treatment effects when there exist mixed categorical and continuous covariates. One distinguishing feature of the approach presented

Efficient Estimation of Average Treatment Effects Using the Estimated Propensity Score

It is shown that weighting with the inverse of a nonparametric estimate of the propensity Score, rather than the true propensity score, leads to efficient estimates of the various average treatment effects, whether the pre-treatment variables have discrete or continuous distributions.

An alternative method of cross-validation for the smoothing of density estimates

An alternative method of cross-validation, based on integrated squared error, recently also proposed by Rudemo (1982), is derived, and Hall (1983) has established the consistency and asymptotic optimality of the new method.

On Smoothing Sparse Multinomial Data

Summary Asymptotic theory is developed for the problem of smoothing sparse multinomial data, with emphasis on the criterion of mean summed square error of estimators of the probability mass