Latent variable graphical model selection via convex optimization

  title={Latent variable graphical model selection via convex optimization},
  author={Venkat Chandrasekaran and Pablo A. Parrilo and Alan S. Willsky},
  journal={2010 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton)},
Suppose we have samples of a subset of a collection of random variables. No additional information is provided about the number of latent variables, nor of the relationship between the latent and observed variables. Is it possible to discover the number of hidden components, and to learn a statistical model over the entire collection of variables? We address this question in the setting in which the latent and observed variables are jointly Gaussian, with the conditional statistics of the… 

Figures from this paper

Learning Exponential Family Graphical Models with Latent Variables using Regularized Conditional Likelihood

A new convex relaxation framework based on regularized conditional likelihood for latent-variable graphical modeling in which the conditional distribution of the observed variables conditioned on the latent variables is given by an exponential family graphical model.

Graphical Model Selection for Gaussian Conditional Random Fields in the Presence of Latent Variables

A method that decomposes the parameters of a conditional Markov random field into the sum of a sparse and a low-rank matrix is suggested and it is derived that is well-behaved in the high-dimensional regime as well as “sparsistent” (i.e., capable of recovering the graph structure).

Learning Latent Variable Dynamic Graphical Models by Confidence Sets Selection

A new method is proposed, which accounts for the uncertainty in the estimation by computing a “confidence neighborhood” containing the true model with a prescribed probability, which allows for the presence of a small number of latent variables in order to enforce sparsity of the identified graph.

Interpreting latent variables in factor models via convex optimization

This paper describes a systematic approach for identifying auxiliary covariates of latent variables in a factor model based on solving computationally tractable convex optimization problems, and it can be viewed as a generalization of the minimum-trace factor analysis procedure for fitting factor models via conveX optimization.

Efficient Latent Variable Graphical Model Selection via Split Bregman Method

This work presents an efficient first-order method based on split Bregman to solve the convex problem and shows that it is significantly faster than the state-of-the-art algorithm on both artificial and real-world data.

Learning the effect of latent variables in Gaussian Graphical models with unobserved variables

This work proposes a convex optimization formulation based on structured matrix sparsity to estimate the complete connectivity of the complete graph including unobserved variables, given the knowledge of the number of missing variables, and a priori knowledge of their level of connectivity.

Ising Models with Latent Conditional Gaussian Variables

This work proposes to learn a sparse + low-rank decomposition of the parameters of an Ising model using a convex regularized likelihood problem and shows that the same problem can be obtained as the dual of a maximum-entropy problem with a new type of relaxation, where the sample means collectively need to match the expected values only up to a given tolerance.

Sparse inverse covariance estimation in Gaussian graphical models

This thesis introduces practically useful advances in structure learning for Gaussian graphical models and their extensions with the addition of latent variables, a non-Gaussian extension, (temporal) conditional mixtures, and methods for efficient inference in a Bayesian formulation.

Spectral Methods for Learning Multivariate Latent Tree Structure

This work considers the problem of learning the structure of multivariate linear tree models, which include a variety of directed tree graphical models with continuous, discrete, and mixed latent

Graphical model inference with unobserved variable via latent tree aggregation

This article provides a model based on spanning trees and the EM algorithm which accounts both for the influence of unobserved variables and the low density of the network, and treats the graph structure and the unobserved nodes as latent variables and compute posterior probabilities of edge appearance.



Identifiability of parameters in latent structure models with many observed variables

A general approach for establishing identifiability utilizing algebraic arguments is demonstrated, which sheds light on the properties of finite mixtures of Bernoulli products, which have been used for decades despite being known to have nonidentifiable parameters.

High-dimensional graphs and variable selection with the Lasso

It is shown that neighborhood selection with the Lasso is a computationally attractive alternative to standard covariance selection for sparse high-dimensional graphs and is hence equivalent to variable selection for Gaussian linear models.

Model Selection Through Sparse Maximum Likelihood Estimation for Multivariate Gaussian or Binary Data

This work considers the problem of estimating the parameters of a Gaussian or binary distribution in such a way that the resulting undirected graphical model is sparse, and presents two new algorithms for solving problems with at least a thousand nodes in the Gaussian case.

Information-Theoretic Limits of Selecting Binary Graphical Models in High Dimensions

The information-theoretic limitations of the problem of graph selection for binary Markov random fields under high-dimensional scaling, in which the graph size and the number of edges k, and/or the maximal node degree d, are allowed to increase to infinity as a function of the sample size n, are analyzed.

On Model Selection Consistency of Lasso

It is proved that a single condition, which is called the Irrepresentable Condition, is almost necessary and sufficient for Lasso to select the true model both in the classical fixed p setting and in the large p setting as the sample size n gets large.

Efficient Computation of ℓ1 Regularized Estimates in Gaussian Graphical Models

An effcient algorithm can be used to efficiently approximate the entire solution path of the ℓ1 regularized maximum likelihood estimates, which also facilitates the choice of tuning parameter.

Model selection and estimation in the Gaussian graphical model

The implementation of the penalized likelihood methods for estimating the concentration matrix in the Gaussian graphical model is nontrivial, but it is shown that the computation can be done effectively by taking advantage of the efficient maxdet algorithm developed in convex optimization.

Nonparametric estimation of large covariance matrices of longitudinal data

Estimation of an unstructured covariance matrix is difficult because of its positive-definiteness constraint. This obstacle is removed by regressing each variable on its predecessors, so that

"Ideal Parent" Structure Learning for Continuous Variable Bayesian Networks

This work presents a general method for speeding structure search for continuous variable networks with common parametric distributions, and naturally and efficiently facilitates the addition of useful new hidden variables into the network structure, a task that is typically considered both conceptually difficult and computationally prohibitive.

Assessing the Validity Domains of Graphical Gaussian Models in Order to Infer Relationships among Components of Complex Biological Systems

This study evaluated the validity domain of each statistical methods recently published from wide-ranging simulated datasets and illustrated their results using recently published biological data.