High-dimensional covariance estimation by minimizing 1-penalized log-determinant divergence

Abstract

Given i.i.d. observations of a random vector X ∈ R p , we study the problem of estimating both its covariance matrix Σ * , and its inverse covariance or concentration matrix Θ * = (Σ *) −1. When X is multivari-ate Gaussian, the non-zero structure of Θ * is specified by the graph of an associated Gaussian Markov random field; and a popular estimator for such sparse Θ * is the ℓ 1-regularized Gaussian MLE. This estimator is sensible even for for non-Gaussian X, since it corresponds to minimizing an ℓ 1-penalized log-determinant Bregman divergence. We analyze its performance under high-dimensional scaling, in which the number of nodes in the graph p, the number of edges s, and the maximum node degree d, are allowed to grow as a function of the sample size n. In addition to the parameters (p, s, d), our analysis identifies other key quantities that control rates: (a) the ℓ∞-operator norm of the true covariance matrix Σ * ; and (b) the ℓ∞ operator norm of the sub-matrix Γ * SS , where S indexes the graph edges, and Γ * = (Θ *) −1 ⊗ (Θ *) −1 ; and (c) a mutual incoherence or irrep-resentability measure on the matrix Γ * and (d) the rate of decay 1/f (n, δ) on the probabilities {| Σ n ij − Σ * ij | > δ}, where Σ n is the sample covariance based on n samples. Our first result establishes consistency of our estimate Θ in the elementwise maximum-norm. This in turn allows us to derive convergence rates in Frobenius and spectral norms, with improvements upon existing results for graphs with maximum node degrees d = o(√ s). In our second result, we show that with probability converging to one, the estimate Θ correctly specifies the zero pattern of the concentration matrix Θ *. We illustrate our theoretical results via simulations for various graphs and problem parameters, showing good correspondences between the theoretical predictions and behavior in simulations.

9 Figures and Tables

Showing 1-10 of 33 references

Highdimensional covariance estimation: Convergence rates of ℓ 1 -regularized logdeterminant divergence

  • P Ravikumar, M J Wainwright, G Raskutti, B Yu
  • 2008
Showing 1-10 of 181 extracted citations
02040602008200920102011201220132014201520162017
Citations per Year

279 Citations

Semantic Scholar estimates that this publication has received between 224 and 352 citations based on the available data.

See our FAQ for additional information.