• Corpus ID: 239050124

Sampling from Arbitrary Functions via PSD Models

@inproceedings{MarteauFerey2022SamplingFA,
  title={Sampling from Arbitrary Functions via PSD Models},
  author={Ulysse Marteau-Ferey and Francis R. Bach and Alessandro Rudi},
  booktitle={AISTATS},
  year={2022}
}
In many areas of applied statistics and machine learning, generating an arbitrary number of independent and identically distributed (i.i.d.) samples from a given distribution is a key task. When the distribution is known only through evaluations of the density, current methods either scale badly with the dimension or require involved implementations. Instead, we take a two-step approach by first modeling the probability distribution and then sampling from that model. We use the recently… 

Figures from this paper

PSD Representations for Effective Probability Models

This work characterize both approximation and generalization capabilities of PSD models, showing that they enjoy strong theoretical guarantees and can perform both sum and product rule in closed form via matrix operations, enjoying the same versatility of mixture models.

SECOND ORDER CONDITIONS TO DECOMPOSE SMOOTH FUNCTIONS AS SQUARES

Second order sufficient conditions in order for a p times continuously differentiable non-negative function to be a sum of squares of p − 2 di-erentiable functions are shown, showing that, locally, the function grows quadratically in directions which are orthogonal to its set of zeros.

Second order conditions to decompose smooth functions as sums of squares

Second order sufficient conditions are shown in order for a p times continuously differentiable non-negative function to be a sum of squares of p− 2 differentiable functions and it is shown that, locally, the function grows quadratically in directions which are orthogonal to its set of zeros.

References

SHOWING 1-10 OF 38 REFERENCES

PSD Representations for Effective Probability Models

This work characterize both approximation and generalization capabilities of PSD models, showing that they enjoy strong theoretical guarantees and can perform both sum and product rule in closed form via matrix operations, enjoying the same versatility of mixture models.

Optimal Rates for the Regularized Least-Squares Algorithm

A complete minimax analysis of the problem is described, showing that the convergence rates obtained by regularized least-squares estimators are indeed optimal over a suitable class of priors defined by the considered kernel.

Non-parametric Models for Non-negative Functions

This paper provides the first model for non-negative functions which benefits from the same good properties of linear models, and proves that it admits a representer theorem and provide an efficient dual formulation for convex problems.

Generalization Properties of Learning with Random Features

The results shed light on the statistical computational trade-offs in large scale kernelized learning, showing the potential effectiveness of random features in reducing the computational complexity while keeping optimal generalization properties.

FALKON: An Optimal Large Scale Kernel Method

This paper proposes FALKON, a novel algorithm that allows to efficiently process millions of points, derived combining several algorithmic principles, namely stochastic subsampling, iterative solvers and preconditioning.

A Dimension-free Computational Upper-bound for Smooth Optimal Transport Estimation

A statistical estimator of smooth optimal transport which achieves a precision ε from ˜ O ( ε − 2 ) independent and identically distributed samples from the distributions, for a computational cost of O when the smoothness increases, hence yielding dimension-free statistical and computational rates, with potentially exponentially dimension-dependent constants.

Less is More: Nyström Computational Regularization

A simple incremental variant of Nystrom Kernel Regularized Least Squares is suggested, where the subsampling level implements a form of computational regularization, in the sense that it controls at the same time regularization and computations.

Minimax Estimation of Maximum Mean Discrepancy with Radial Kernels

The first known lower bounds for the estimation of MMD based on finite samples are presented and established, establishing the minimax rate optimality of the empirical estimator and its U-statistic variant, which are usually employed in applications.

Hilbert Space Embeddings and Metrics on Probability Measures

It is shown that the distance between distributions under γk results from an interplay between the properties of the kernel and the distributions, by demonstrating that distributions are close in the embedding space when their differences occur at higher frequencies.

Globally Convergent Newton Methods for Ill-conditioned Generalized Self-concordant Losses

This is the first large-scale algorithm to solve logistic and softmax regressions in the non-parametric setting with large condition numbers and theoretical guarantees, generalizing the results known for least-squares regression.