Marginal Likelihoods for Distributed Parameter Estimation of Gaussian Graphical Models

  title={Marginal Likelihoods for Distributed Parameter Estimation of Gaussian Graphical Models},
  author={Zhaoshi Meng and Dennis L. Wei and A. Wiesel and A. Hero},
  journal={IEEE Transactions on Signal Processing},
We consider distributed estimation of the inverse covariance matrix, also called the concentration or precision matrix, in Gaussian graphical models. Traditional centralized estimation often requires global inference of the covariance matrix, which can be computationally intensive in large dimensions. Approximate inference based on message-passing algorithms, on the other hand, can lead to unstable and biased estimation in loopy graphical models. Here, we propose a general framework for… Expand
Approximate Bayesian estimation in large coloured graphical Gaussian models
Distributed estimation methods have recently been used to compute the maximum likelihood estimate of the precision matrix for large graphical Gaussian models. Our aim, in this paper, is to give aExpand
Local conditional and marginal approach to parameter estimation in discrete graphical models
The purpose of this paper is to show that, when all the neighbors of v are linked to other nodes in the graph, the estimates obtained through local conditional and marginal likelihoods are identical, and the two MCLE are usually very close. Expand
Maximum Likelihood Estimate in Discrete Hierarchical Log-Linear Models
Hierarchical log-linear models are essential tools used for relationship identification between variables in complex high-dimensional problems. In this thesis we study two problems: the computationExpand
Bayesian Estimation of Graphical Gaussian Models with Edges and Vertices Symmetries
In this thesis, we consider the Bayesian analysis of undirected graphical Gaussian models with edges and vertices symmetries. The graphical Gaussian models with equality constraints on the precisionExpand
Communication-efficient Distributed Estimation and Inference for Transelliptical Graphical Models
We propose communication-efficient distributed estimation and inference methods for the transelliptical graphical model, a semiparametric extension of the elliptical distribution in the highExpand
Approximate computation of the posterior mean in large coloured graphical Gaussian models April 25 , 2016
  • 2016
In this paper, we consider graphical Gaussian models with symmetry constraints. Symmetry restrictions in the multivariate Gaussian have a long history dating back to Wilks (1946) and the reader isExpand
Efficient distributed estimation of inverse covariance matrices
This paper proposes a method where, in a single round of communication, each machine transfers a small subset of the entries of the inverse covariance matrix, and shows that, with this efficient distributed method, the error rates can be comparable with estimation in a non-distributed setting. Expand
Learning of Tree-Structured Gaussian Graphical Models on Distributed Data Under Communication Constraints
This paper presents a set of communication-efficient strategies, which are theoretically proved to convey sufficient information for reliable learning of the structure of tree-structured Gaussian graphical models from distributed data. Expand
Penalized composite likelihood for colored graphical Gaussian models
It is shown that the proposed penalized composite likelihood method performs well in both the selection of nonzero elements in the precision matrix and the identification of symmetry structures in graphical models. Expand
Distributed Parameter Estimation in Probabilistic Graphical Models
A general condition on composite likelihood decompositions of these models which guarantees the global consistency of distributed estimators, provided the local estimators are consistent is introduced. Expand


Distributed Learning of Gaussian Graphical Models via Marginal Likelihoods
This paper proposes a general framework for distributed estimation based on a maximum marginal likelihood (MML) approach, and derives and considers solving a convex relaxation of the MML problem, and proves that this relaxed MML estimator is asymptotically consistent. Expand
Distributed Covariance Estimation in Gaussian Graphical Models
This work proposes to improve the MSE performance by introducing additional symmetry constraints using averaging and pseudolikelihood estimation approaches, and compute the proposed estimates using message passing protocols, which can be efficiently implemented in large scale graphical models with many nodes. Expand
Marginal likelihoods for distributed estimation of graphical model parameters
An alternative framework for distributed parameter estimation based on maximizing marginal likelihoods is described, and an asymptotic analysis of the proposed estimator is provided, deriving in particular its rate of convergence. Expand
Distributed parameter estimation of discrete hierarchical models via marginal likelihoods
We consider discrete graphical models Markov with respect to a graph $G$ and propose two distributed marginal methods to estimate the maximum likelihood estimate of the canonical parameter of theExpand
High-dimensional Sparse Inverse Covariance Estimation using Greedy Methods
Surprisingly, it is shown that both the local and global greedy methods learn the full structure of the model with high probability given just O(d log(p)) samples, which is a significant improvement over state of the art ‘1-regularized Gaussian MLE (Graphical Lasso) that requires O( d 2 log( p)) samples. Expand
Convex optimization techniques for fitting sparse Gaussian graphical models
This work considers the problem of fitting a large-scale covariance matrix to multivariate Gaussian data in such a way that the inverse is sparse, thus providing model selection, and presents two new algorithms aimed at solving problems with a thousand nodes. Expand
High-dimensional covariance estimation by minimizing ℓ1-penalized log-determinant divergence
Given i.i.d. observations of a random vector X 2 R p , we study the problem of estimating both its covariance matrix � ∗ , and its inverse covariance or concentration matrix � ∗ = (� ∗ ) −1 . WeExpand
Estimating the "Wrong" Graphical Model: Benefits in the Computation-Limited Setting
  • M. Wainwright
  • Mathematics, Computer Science
  • J. Mach. Learn. Res.
  • 2006
The key result of this paper is that in the computation-limited setting, using an inconsistent parameter estimator is provably beneficial, since the resulting errors can partially compensate for errors made by using an approximate prediction technique. Expand
Fisher Information in Gaussian Graphical Models
This note summarizes various derivations, formulas and computational algorithms relevant to the Fisher information matrix of Gaussian graphical models with respect to either an exponentialExpand
Graphical Models, Exponential Families, and Variational Inference
The variational approach provides a complementary alternative to Markov chain Monte Carlo as a general source of approximation methods for inference in large-scale statistical models. Expand