Corpus ID: 222090989

Analysis of KNN Density Estimation

  title={Analysis of KNN Density Estimation},
  author={Puning Zhao and Lifeng Lai},
We analyze the $\ell_1$ and $\ell_\infty$ convergence rates of k nearest neighbor density estimation method. Our analysis includes two different cases depending on whether the support set is bounded or not. In the first case, the probability density function has a bounded support and is bounded away from zero. We show that kNN density estimation is minimax optimal under both $\ell_1$ and $\ell_\infty$ criteria, if the support set is known. If the support set is unknown, then the convergence… Expand

Figures and Tables from this paper

Active Covering
This work analyzes the problem of active covering, where the learner is given an unlabeled dataset and can sequentially label query examples and provides a simple active learning method that attains an improved excess query cost of Õ(n(D−1)/D). Expand
Label Smoothed Embedding Hypothesis for Out-of-Distribution Detection
This work proposes an unsupervised method to detect out-of-distribution samples using a k-NN density estimate with respect to a classification model’s intermediate activations on indistribution samples, and leverages a recent insight about label smoothing. Expand


Minimax Rate Optimal Adaptive Nearest Neighbor Classification and Regression
The convergence rate of the proposed adaptive kNN method is characterized, and it is shown that it matches the minimax lower bound. Expand
Uniform Convergence Rate of the Kernel Density Estimator Adaptive to Intrinsic Volume Dimension
The volume dimension is proposed, called the volume dimension, to measure the intrinsic dimension of the support of a probability distribution based on the rates of decay of the probability of vanishing Euclidean balls and is useful for problems in geometric inference and topological data analysis. Expand
Uniform Convergence Rates for Kernel Density Estimation
Finite-sample high-probability density estimation bounds for multivariate KDE are derived under mild density assumptions which hold uniformly in x ∈ R and bandwidth matrices and uniform convergence results for local intrinsic dimension estimation are given. Expand
Rates of strong uniform consistency for multivariate kernel density estimators
Abstract Let fn denote the usual kernel density estimator in several dimensions. It is shown that if {an} is a regular band sequence, K is a bounded square integrable kernel of several variables,Expand
On minimax density estimation on R
The problem of density estimation on R on the basis of an independent sample X1 ..., XN with common density f is discussed. The behaviour of the minimax Lp risk, 1 < p < oc, is studied when f belongsExpand
Density estimation by wavelet thresholding
Density estimation is a commonly used test case for nonparametric estimation methods. We explore the asymptotic properties of estimators based on thresholding of empirical wavelet coefficients.Expand
The $L_1$ Convergence of Kernel Density Estimates
supB e|e1n(B) v(B)l -*,nO in probability (or w.p. 1), where XI, ** Xn are independent, identically distributed with an arbitrary probability measure P on `, Pn is the empircal measure for X1, ** XnExpand
Rate of strong uniform convergence of k-NN density estimates
Abstract Let f n (x) be the univariate k -nearest neighbor ( k -NN) density estimate proposed by Loftsgaarden and Quesenberry (1965). By using similar techniques as in Bahadur's representation ofExpand
Modal-set estimation with an application to clustering
A first procedure that can estimate -- with statistical consistency guarantees -- any local-maxima of a density, under benign distributional conditions is presented, and is shown to be competitive on clustering applications, and moreover is quite stable to a wide range of settings of its tuning parameter. Expand
Classification Based on Hybridization of Parametric and Nonparametric Classifiers
This paper uses some simulated examples and benchmark data sets to examine the performance of these hybrid discriminant analysis tools and combines their strengths to develop some hybrid classification methods. Expand