High-dimensional covariance estimation by minimizing 1-penalized log-determinant divergence

Abstract

Given i.i.d. observations of a random vector X ∈ R, we study the problem of estimating both its covariance matrix Σ, and its inverse covariance or concentration matrix Θ = (Σ). When X is multivariate Gaussian, the non-zero structure of Θ is specified by the graph of an associated Gaussian Markov random field; and a popular estimator for such sparse Θ is the l1-regularized Gaussian MLE. This estimator is sensible even for for non-Gaussian X, since it corresponds to minimizing an l1-penalized log-determinant Bregman divergence. We analyze its performance under high-dimensional scaling, in which the number of nodes in the graph p, the number of edges s, and the maximum node degree d, are allowed to grow as a function of the sample size n. In addition to the parameters (p, s, d), our analysis identifies other key quantities that control rates: (a) the l∞-operator norm of the true covariance matrix Σ; and (b) the l∞ operator norm of the sub-matrix Γ∗SS , where S indexes the graph edges, and Γ∗ = (Θ) ⊗ (Θ); and (c) a mutual incoherence or irrepresentability measure on the matrix Γ∗ and (d) the rate of decay 1/f(n, δ) on the probabilities {|Σ̂ij − Σ∗ij | > δ}, where Σ̂ is the sample covariance based on n samples. Our first result establishes consistency of our estimate Θ̂ in the elementwise maximum-norm. This in turn allows us to derive convergence rates in Frobenius and spectral norms, with improvements upon existing results for graphs with maximum node degrees d = o( √ s). In our second result, we show that with probability converging to one, the estimate Θ̂ correctly specifies the zero pattern of the concentration matrix Θ∗. We illustrate our theoretical results via simulations for various graphs and problem parameters, showing good correspondences between the theoretical predictions and behavior in simulations. AMS 2000 subject classifications: Primary 62F12; secondary 62F30.

View Slides

9 Figures and Tables

0204060200920102011201220132014201520162017
Citations per Year

204 Citations

Semantic Scholar estimates that this publication has 204 citations based on the available data.

See our FAQ for additional information.

Cite this paper

@inproceedings{Ravikumar2009HighdimensionalCE, title={High-dimensional covariance estimation by minimizing 1-penalized log-determinant divergence}, author={Pradeep Ravikumar and Martin J. Wainwright and Garvesh Raskutti and Bin Yu}, year={2009} }