• Corpus ID: 2542875

Convergence guarantees for kernel-based quadrature rules in misspecified settings

@inproceedings{Kanagawa2016ConvergenceGF,
  title={Convergence guarantees for kernel-based quadrature rules in misspecified settings},
  author={Motonobu Kanagawa and Bharath K. Sriperumbudur and Kenji Fukumizu},
  booktitle={NIPS},
  year={2016}
}
Kernel-based quadrature rules are becoming important in machine learning and statistics, as they achieve super-$\sqrt{n}$ convergence rates in numerical integration, and thus provide alternatives to Monte Carlo integration in challenging settings where integrands are expensive to evaluate or where integrands are high dimensional. These rules are based on the assumption that the integrand has a certain degree of smoothness, which is expressed as that the integrand belongs to a certain… 

Figures from this paper

Convergence Analysis of Deterministic Kernel-Based Quadrature Rules in Misspecified Settings
TLDR
This paper reveals a condition on design points to make Bayesian quadrature robust to misspecification, and shows that it may adaptively achieve the optimal rate of convergence in the Sobolev space of a lesser order under a slightly stronger regularity condition on the integrand.
An advantage of kernel quadrature rules is that the nodes are not prescribed
TLDR
This article shows that the weights of a kernel quadrature rule can be computed efficiently and exactly for up to tens of millions of nodes if the kernel, integration domain, and measure are fully symmetric and the node set is a union of fully symmetrical sets.
On the Sampling Problem for Kernel Quadrature
TLDR
It is argued that the practical choice of sampling distribution is an important open problem and a novel automatic approach based on adaptive tempering and sequential Monte Carlo is considered, demonstrating a dramatic reduction in integration error.
Kernel interpolation with continuous volume sampling
TLDR
This work introduces and analyse continuous volume sampling (VS), the continuous counterpart -- for choosing node locations -- of a discrete distribution introduced in (Deshpande & Vempala, 2006), and proves almost optimal bounds for interpolation and quadrature under VS.
Order-Optimal Error Bounds for Noisy Kernel-Based Bayesian Quadrature
TLDR
It is found that when the black-box queries are subject to Gaussian noise having variance σ, any algorithm making at most T queries must incur a mean absolute error of Ω, and there exists a non-adaptive algorithm attaining an error of at most O.
Convergence Guarantees for Adaptive Bayesian Quadrature Methods
TLDR
For a broad class of adaptive Bayesian quadrature methods, this work proves consistency, deriving non-tight but informative convergence rates and introduces a new concept the authors call weak adaptivity.
Fully symmetric kernel quadrature
TLDR
This article shows that the weights of a kernel quadrature rule can be computed efficiently and exactly for up to tens of millions of nodes if the kernel, integration domain, and measure are fully symmetric and the node set is a union of fully symmetrical sets.
On the positivity and magnitudes of Bayesian quadrature weights
TLDR
It is shown that the weights are positive in the univariate case if the design points locally minimise the posterior integral variance and the covariance kernel is totally positive, suggesting that gradient-based optimisation of design points may be effective in constructing stable and robust Bayesian quadrature rules.
Statistical computation with kernels
TLDR
This thesis analyzes a well-known algorithm for numerical integration called Bayesian quadrature, and provides consistency and contraction rates, and studies two minimum distance estimators derived from kernel-based statistical divergences which can be used for unnormalised and generative models.
Convergence rates for a class of estimators based on Stein’s method
Gradient information on the sampling distribution can be used to reduce the variance of Monte Carlo estimators via Stein's method. An important application is that of estimating an expectation of a
...
1
2
3
4
...

References

SHOWING 1-10 OF 32 REFERENCES
On the Equivalence between Kernel Quadrature Rules and Random Feature Expansions
  • F. Bach
  • Computer Science, Mathematics
    J. Mach. Learn. Res.
  • 2017
TLDR
Theoretical analysis of the number of required samples for a given approximation error leads to both upper and lower bounds that are based solely on the eigenvalues of the associated integral operator and match up to logarithmic terms.
Scattered-Data Interpolation on Rn: Error Estimates for Radial Basis and Band-Limited Functions
TLDR
This paper obtains Sobolev-type error estimates on compact regions of $\mathbb R^n$ when the RBFs have Fourier transforms that decay algebraically, and shows that it is possible to construct band-limited interpolants that are also near-best approximants to such functions.
Quasi-Monte Carlo Feature Maps for Shift-Invariant Kernels
TLDR
A new discrepancy measure called box discrepancy is derived based on theoretical characterizations of the integration error with respect to a given sequence based on explicit box discrepancy minimization in Quasi-Monte Carlo (QMC) approximations.
Learning Theory Estimates via Integral Operators and Their Approximations
TLDR
The regression problem in learning theory is investigated with least square Tikhonov regularization schemes in reproducing kernel Hilbert spaces (RKHS) and the sampling operator is applied to the error analysis in both the RKHS norm and the L2 norm.
Frank-Wolfe Bayesian Quadrature: Probabilistic Integration with Theoretical Guarantees
TLDR
This paper presents the first probabilistic integrator that admits such theoretical treatment, called Frank-Wolfe Bayesian Quadrature (FWBQ), which is applied to successfully quantify numerical error in the solution to a challenging Bayesian model choice problem in cellular biology.
Bayes–Hermite quadrature
Explicit Constructions of Quasi-Monte Carlo Rules for the Numerical Integration of High-Dimensional Periodic Functions
  • J. Dick
  • Computer Science, Mathematics
    SIAM J. Numer. Anal.
  • 2007
In this paper, we give explicit constructions of point sets in the $s$-dimensional unit cube yielding quasi-Monte Carlo algorithms which achieve the optimal rate of convergence of the worst-case
Mercer’s Theorem on General Domains: On the Interaction between Measures, Kernels, and RKHSs
Given a compact metric space X and a strictly positive Borel measure ν on X, Mercer’s classical theorem states that the spectral decomposition of a positive self-adjoint integral operator
Control Functionals for Quasi-Monte Carlo Integration
TLDR
This strategy is explored herein and shown to be well-suited to modern statistical computation and to trade-off numerical integration with functional approximation.
Walsh Spaces Containing Smooth Functions and Quasi-Monte Carlo Rules of Arbitrary High Order
  • J. Dick
  • Mathematics, Computer Science
    SIAM J. Numer. Anal.
  • 2008
TLDR
A Walsh space is defined which contains all functions whose partial mixed derivatives up to order $\delta \ge 1$ exist and have finite variation and it is shown that quasi-Monte Carlo rules based on digital $(t,\alpha,s)$-sequences achieve the optimal rate of convergence of the worst-case error for numerical integration.
...
1
2
3
4
...