A recursive procedure for density estimation on the binary hypercube

@article{Raginsky2011ARP,
  title={A recursive procedure for density estimation on the binary hypercube},
  author={Maxim Raginsky and Jorge G. Silva and Svetlana Lazebnik and Rebecca M. Willett},
  journal={arXiv: Statistics Theory},
  year={2011}
}
This paper describes a recursive estimation procedure for multivariate binary densities (probability distributions of vectors of Bernoulli random variables) using orthogonal expansions. For $d$ covariates, there are $2^d$ basis coefficients to estimate, which renders conventional approaches computationally prohibitive when $d$ is large. However, for a wide class of densities that satisfy a certain sparsity condition, our estimator runs in probabilistic polynomial time and adapts to the unknown… 

Figures from this paper

An exploration of the role of principal inertia components in information theory
TLDR
It is shown that, under certain symmetry conditions, the principal inertia components play an important role in estimating one-bit functions of X, namely f(X), given an observation of Y, which naturally leads to the conjecture that the mutual information between f( X) and Y is maximized when all the principal momentum components have equal value.
Principal Inertia Components and Applications
TLDR
It is proved that the PICs are related to the fundamental limits of perfect privacy, and they can be used to characterize information-theoretic limits of certain estimation problems.

References

SHOWING 1-10 OF 45 REFERENCES
Density estimation by wavelet thresholding
Density estimation is a commonly used test case for nonparametric estimation methods. We explore the asymptotic properties of estimators based on thresholding of empirical wavelet coefficients.
Minimax Bayes, Asymptotic Minimax and Sparse Wavelet Priors
Pinsker (1980) gave a precise asymptotic evaluation of the minimax mean squared error of estimation of a signal in Gaussian noise when the signal is known a priori to lie in a compact ellipsoid in
Modern statistical estimation via oracle inequalities
  • E. Candès
  • Mathematics, Computer Science
    Acta Numerica
  • 2006
TLDR
This survey paper aims at reconstructing the history of how thresholding rules came to be popular in statistics and describing, in a not overly technical way, the domain of their application.
Adaptive estimation of the intensity of inhomogeneous Poisson processes via concentration inequalities
In this paper, we establish oracle inequalities for penalized projection estimators of the intensity of an inhomogeneous Poisson process. We study consequently the adaptive properties of penalized
Nonparametric Curve Estimation
TLDR
H hierarchies of kernels offer large possibilities for optimizing smoothers such as in Cross-Validation techniques, double or multiple kernel procedures, multiparameter kernel estimation, reduction of kernel complexity and many others which are far from being fully investigated.
Learning decision trees using the Fourier spectrum
TLDR
The authors demonstrate that any functionf whose L -norm is polynomial can be approximated by a polynomially sparse function, and prove that boolean decision trees with linear operations are a subset of this class of functions.
Near-optimal sparse fourier representations via sampling
TLDR
An algorithm for finding a Fourier representation of B for a given discrete signal signal A, such that A is within the factor (1 +ε) of best possible $\|\signal-\repn_\opt\|_2^2$.
Block threshold rules for curve estimation using kernel and wavelet methods
TLDR
It is argued that block thresholding has a number of advantages, including that it produces adaptive estimators which achieve minimax-optimal convergence rates without the logarithmic penalty that is sometimes associated with term-by-term thresholding.
Numerical performance of block thresholded wavelet estimators
TLDR
It is shown that in the context of moderate to low signal-to-noise ratios, this ‘block thresholding’ approach does indeed improve performance, by allowing greater adaptivity and reducing mean squared error.
On the fourier tails of bounded functions over the discrete cube
TLDR
This work handles all bounded functions, at the price of a much faster tail decay, and the rate of decay is shown to be both roughly necessary and sufficient.
...
...