Fast Spatial Gaussian Process Maximum Likelihood Estimation via Skeletonization Factorizations

  title={Fast Spatial Gaussian Process Maximum Likelihood Estimation via Skeletonization Factorizations},
  author={Victor Minden and Anil Damle and Kenneth L. Ho and Lexing Ying},
  journal={Multiscale Model. Simul.},
Maximum likelihood estimation for parameter fitting given observations from a Gaussian process in space is a computationally demanding task that restricts the use of such methods to moderately sized datasets. We present a framework for unstructured observations in two spatial dimensions that allows for evaluation of the log-likelihood and its gradient (i.e., the score equations) in $\tilde O(n^{3/2})$ time under certain assumptions, where $n$ is the number of observations. Our method relies on… Expand
Tucker Tensor Analysis of Matérn Functions in Spatial Statistics
Abstract In this work, we describe advanced numerical tools for working with multivariate functions and for the analysis of large data sets. These tools will drastically reduce the required computingExpand
Computationally efficient spatial modeling using recursive skeletonization factorizations
Abstract Recursive skeletonization factorization techniques can evaluate an accurate approximation to the log-likelihood for irregularly sited two-dimensional spatial data under a Gaussian process inExpand
Scalable Gaussian Process Computations Using Hierarchical Matrices
A kernel-independent method that applies hierarchical matrices to the problem of maximum likelihood estimation for Gaussian processes provides natural and scalable stochastic estimators for its gradient and Hessian as well as the expected Fisher information matrix that are computable in quasilinear complexity for a large range of models. Expand
Linear-Cost Covariance Functions for Gaussian Random Fields
Gaussian random fields (GRF) are a fundamental stochastic model for spatiotemporal data analysis. An essential ingredient of GRF is the covariance function that characterizes the joint GaussianExpand
Fast sampling of parameterised Gaussian random fields
A reduced basis surrogate built from snapshots of Karhunen–Loeve eigenvectors is constructed and a linearisation of the covariance function is suggested and described and the associated online–offline decomposition is described. Expand
Certified and fast computations with shallow covariance kernels
This work introduces and analyze a new and certified algorithm for the low-rank approximation of a parameterized family of covariance operators which represents an extension of the adaptive cross approximation method for symmetric positive definite matrices. Expand
Kryging: Geostatistical analysis of large-scale datasets using Krylov subspace methods
Analyzing massive spatial datasets using Gaussian process model poses computational challenges. This is a problem prevailing heavily in applications such as environmental modeling, ecology, forestryExpand
Efficient reduced-rank methods for Gaussian processes with eigenfunction expansions
A reduced-rank algorithm for Gaussian process regression and a class of fast algorithms for Bayesian fitting of hyperparameters, and demonstrate the performance of the algorithms with numerical experiments in one and two dimensions. Expand
A general framework for Vecchia approximations of Gaussian processes
Gaussian processes (GPs) are commonly used as models for functions, time series, and spatial fields, but they are computationally infeasible for large datasets. Focusing on the typical setting ofExpand
FLAM: Fast Linear Algebra in MATLAB - Algorithms for Hierarchical Matrices
Many large matrices in science and engineering possess a special hierarchical low-rank structure that enables fast multiplication and inversion, among other fundamental operations. Such matricesExpand


Fast Direct Methods for Gaussian Processes
This work shows that for the most commonly used covariance functions, the matrix C can be hierarchically factored into a product of block low-rank updates of the identity matrix, yielding an O(n log2 n) algorithm for inversion and enables the evaluation of the determinant det(C), permitting the direct calculation of probabilities in high dimensions under fairly broad assumptions on the kernel defining K. Expand
A Matrix-free Approach for Solving the Parametric Gaussian Process Maximum Likelihood Problem
This work presents a matrix-free approach for computing the solution of the maximum likelihood problem involving Gaussian processes based on a stochastic programming reformulation followed by sample average approximation applied to either the maximization problem or its optimality conditions. Expand
An explicit link between Gaussian fields and Gaussian Markov random fields: the stochastic partial differential equation approach
Continuously indexed Gaussian fields (GFs) are the most important ingredient in spatial statistical modelling and geostatistics. The specification through the covariance function gives an intuitiveExpand
Approximating likelihoods for large spatial data sets
Likelihood methods are often difficult to use with large, irregularly sited spatial data sets, owing to the computational burden. Even for Gaussian models, exact calculations of the likelihood forExpand
Approximating Gaussian Processes with H2-Matrices
The kernel matrix is replaced by a data-sparse approximation, called an ${\mathcal H^2$-matrix, which leads to significant reductions in storage requirements and computing times for large data sets in lower dimensional spaces. Expand
Multi-Level Restricted Maximum Likelihood Covariance Estimation and Kriging for Large Non-Gridded Spatial Datasets
We develop a multi-level restricted Gaussian maximum likelihood method for estimating the covariance function parameters and computing the best unbiased predictor. Our approach produces a new set ofExpand
A full scale approximation of covariance functions for large spatial data sets
Summary.  Gaussian process models have been widely used in spatial statistics but face tremendous computational challenges for very large data sets. The model fitting and spatial prediction of suchExpand
A Fast Direct Solver for Structured Linear Systems by Recursive Skeletonization
Using the recently developed interpolative decomposition of a low-rank matrix in a recursive manner, an approximation of the original matrix is embedded into a larger but highly structured sparse one that allows fast factorization and application of the inverse. Expand
Application of hierarchical matrices for computing the Karhunen–Loève expansion
A log-linear computational cost of the matrix-vector product and a log- linear storage requirement yield an efficient and fast discretisation of the random fields presented. Expand
A stochastic estimator of the trace of the influence matrix for laplacian smoothing splines
An unbiased stochastic estimator of tr(I-A), where A is the influence matrix associated with the calculation of Laplacian smoothing splines, is described. The estimator is similar to one recentlyExpand