Denoising Score Matching via Random Fourier Features

@article{Olga2021DenoisingSM,
  title={Denoising Score Matching via Random Fourier Features},
  author={Tsimboy Olga and Yermek Kapushev and Evgeny Burnaev and I. Oseledets},
  journal={IEEE Access},
  year={2021},
  volume={10},
  pages={34154-34169}
}
The density estimation is one of the core problems in statistics. Despite this, existing techniques like maximum likelihood estimation are computationally inefficient in case of complex parametric families due to the intractability of the normalizing constant. For this reason, an interest in score matching has increased, being independent on the normalizing constant. However, such an estimator is consistent only for distributions with the full space support. One of the approaches to make it… 
[PDF] Semantic Reader

Figures and Tables from this paper

References

SHOWING 1-10 OF 40 REFERENCES

Efficient and principled score estimation with Nyström kernel exponential families

Compared to an existing score learning approach using a denoising autoencoder, the estimator is empirically more data-efficient when estimating the score, runs faster, and has fewer parameters (which can be tuned in a principled and interpretable way), in addition to providing statistical guarantees.

Estimation of Non-Normalized Statistical Models by Score Matching

While the estimation of the gradient of log-density function is, in principle, a very difficult non-parametric problem, it is proved a surprising result that gives a simple formula that simplifies to a sample average of a sum of some derivatives of the log- density given by the model.

Kernel Exponential Family Estimation via Doubly Dual Embedding

A connection between kernel exponential family estimation and MMD-GANs is established, revealing a new perspective for understanding GANs and it is shown that the proposed estimator empirically outperforms state-of-the-art estimators.

Towards a Unified Analysis of Random Fourier Features

This work provides the first unified risk analysis of learning with random Fourier features using the squared error and Lipschitz continuous loss functions and devise a simple approximation scheme which provably reduces the computational cost without loss of statistical efficiency.

Linear estimating equations for exponential families with application to Gaussian linear concentration models

Some extensions of score matching

Graphical Models for Non-Negative Data Using Generalized Score Matching

This paper gives a generalized form of score matching for non-negative data that improves estimation efficiency and generalizes the regularized score matching method of Lin et al. (2016) fornon-negative Gaussian graphical models, with improved theoretical guarantees.

Orlicz Random Fourier Features

A finite-sample deviation bound is established for a general class of polynomial-growth functions under α-exponential Orlicz condition on the distribution of the sample for RFFs, which implies a.s. convergence with tight rate for arbitrary kernel with α- expansions spectrum and any order of derivative.

Estimation of High-Dimensional Graphical Models Using Regularized Score Matching.

It is confirmed that regularized score matching achieves state-of-the-art performance in the Gaussian case and provides a valuable tool for computationally efficient estimation in non-Gaussian graphical models.

Score matching estimators for directional distributions

One of the major problems for maximum likelihood estimation in the well-established directional models is that the normalising constants can be difficult to evaluate. A new general method of "score…