Convergence Analysis of Deterministic Kernel-Based Quadrature Rules in Misspecified Settings

@article{Kanagawa2017ConvergenceAO,
  title={Convergence Analysis of Deterministic Kernel-Based Quadrature Rules in Misspecified Settings},
  author={Motonobu Kanagawa and Bharath K. Sriperumbudur and Kenji Fukumizu},
  journal={Foundations of Computational Mathematics},
  year={2017},
  volume={20},
  pages={155-194}
}
This paper presents convergence analysis of kernel-based quadrature rules in misspecified settings, focusing on deterministic quadrature in Sobolev spaces. In particular, we deal with misspecified settings where a test integrand is less smooth than a Sobolev RKHS based on which a quadrature rule is constructed. We provide convergence guarantees based on two different assumptions on a quadrature rule: one on quadrature weights and the other on design points. More precisely, we show that… 

On the positivity and magnitudes of Bayesian quadrature weights

It is shown that the weights are positive in the univariate case if the design points locally minimise the posterior integral variance and the covariance kernel is totally positive, suggesting that gradient-based optimisation of design points may be effective in constructing stable and robust Bayesian quadrature rules.

Convergence Guarantees for Adaptive Bayesian Quadrature Methods

For a broad class of adaptive Bayesian quadrature methods, this work proves consistency, deriving non-tight but informative convergence rates and introduces a new concept the authors call weak adaptivity.

On Linear Convergence of Weighted Kernel Herding

This work provides a novel convergence analysis of two popular sampling algorithms, Weighted Kernel Herding and Sequential Bayesian Quadrature, and suggests a simplifying assumption that is true for most cases in finite dimensions, and that acts as a sufficient condition for linear convergence to hold in the much harder case of infinite dimensions.

Convergence Guarantees for Gaussian Process Approximations Under Several Observation Models

The main novelty in this paper is that the results cover a wide range of observation models including interpolation, approximation with deterministic corruption and regression with Gaussian noise.

Geometric rates of convergence for kernel-based sampling algorithms

A near-geometric rate of convergence is established of weighted kernel herding and sequential Bayesian quadrature, two kernel-based sampling algorithms for estimating integrals with respect to some target probability measure.

Convergence Guarantees for Gaussian Process Means With Misspecified Likelihoods and Smoothness

This paper describes how the experimental design and choice of kernel and kernel hyperparameters can be adapted to alleviate model misspecification.

Sparse solutions of the kernel herding algorithm by improved gradient approximation

This paper proposes a modified kernel herding algorithm whose framework was introduced in a previous study and aims to obtain sparser solutions while preserving the advantages of standardkernel herding, and provides a new theoretical analysis of the kernel quadrature rules with fully-corrective weights, which realizes faster convergence speeds than those of previous studies.

Maximum likelihood estimation and uncertainty quantification for Gaussian process approximation of deterministic functions

It is shown that the maximum likelihood estimation of the scale parameter alone provides significant adaptation against misspecification of the Gaussian process model in the sense that the model can become "slowly" overconfident at worst, regardless of the difference between the smoothness of the data-generating function and that expected by the model.

Positively Weighted Kernel Quadrature via Subsampling

This approach combines the spectral properties of the kernel with recombination results about point measures and results in effective algorithms that construct convex quadrature rules using only access to i.i.d. samples.

Optimal Monte Carlo integration on closed manifolds

General theoretical results for Sobolev spaces on closed Riemannian manifolds are contributed, where it is verified that the re-weighting of random points yields optimal approximation rates up to a logarithmic factor.

References

SHOWING 1-10 OF 74 REFERENCES

Convergence guarantees for kernel-based quadrature rules in misspecified settings

This work derives convergence rates that depend on the (unknown) lesser smoothness of the integrand, where the degree of smoothness is expressed via powers of RKHSs or via Sobolev spaces.

On the Equivalence between Kernel Quadrature Rules and Random Feature Expansions

  • F. Bach
  • Computer Science, Mathematics
    J. Mach. Learn. Res.
  • 2017
Theoretical analysis of the number of required samples for a given approximation error leads to both upper and lower bounds that are based solely on the eigenvalues of the associated integral operator and match up to logarithmic terms.

Convergence rates for a class of estimators based on Stein’s method

Gradient information on the sampling distribution can be used to reduce the variance of Monte Carlo estimators via Stein's method. An important application is that of estimating an expectation of a

Harder, Better, Faster, Stronger Convergence Rates for Least-Squares Regression

We consider the optimization of a quadratic objective function whose gradients are only accessible through a stochastic oracle that returns the gradient at any given point plus a zero-mean finite

Sobolev Error Estimates and a Bernstein Inequality for Scattered Data Interpolation via Radial Basis Functions

Error estimates for scattered-data interpolation via radial basis functions (RBFs) for target functions in the associated reproducing kernel Hilbert space (RKHS) have been known for a long time.

Active Uncertainty Calibration in Bayesian ODE Solvers

The novel filtering-based method Bayesian Quadrature filtering (BQF) is proposed which uses Bayesian quadrature to actively learn the imprecision in the gradient measurement by collecting multiple gradient evaluations.

Scattered-Data Interpolation on Rn: Error Estimates for Radial Basis and Band-Limited Functions

This paper obtains Sobolev-type error estimates on compact regions of $\mathbb R^n$ when the RBFs have Fourier transforms that decay algebraically, and shows that it is possible to construct band-limited interpolants that are also near-best approximants to such functions.

Quasi-Monte Carlo Feature Maps for Shift-Invariant Kernels

A new discrepancy measure called box discrepancy is derived based on theoretical characterizations of the integration error with respect to a given sequence based on explicit box discrepancy minimization in Quasi-Monte Carlo (QMC) approximations.

Walsh Spaces Containing Smooth Functions and Quasi-Monte Carlo Rules of Arbitrary High Order

  • J. Dick
  • Mathematics, Computer Science
    SIAM J. Numer. Anal.
  • 2008
A Walsh space is defined which contains all functions whose partial mixed derivatives up to order $\delta \ge 1$ exist and have finite variation and it is shown that quasi-Monte Carlo rules based on digital $(t,\alpha,s)$-sequences achieve the optimal rate of convergence of the worst-case error for numerical integration.

Frank-Wolfe Bayesian Quadrature: Probabilistic Integration with Theoretical Guarantees

This paper presents the first probabilistic integrator that admits such theoretical treatment, called Frank-Wolfe Bayesian Quadrature (FWBQ), which is applied to successfully quantify numerical error in the solution to a challenging Bayesian model choice problem in cellular biology.
...