# Message-passing algorithms for quadratic minimization

@article{Ruozzi2013MessagepassingAF, title={Message-passing algorithms for quadratic minimization}, author={Nicholas Ruozzi and Sekhar C. Tatikonda}, journal={J. Mach. Learn. Res.}, year={2013}, volume={14}, pages={2287-2314} }

Gaussian belief propagation (GaBP) is an iterative algorithm for computing the mean (and variances) of a multivariate Gaussian distribution, or equivalently, the minimum of a multivariate positive definite quadratic function. Sufficient conditions, such as walk-summability, that guarantee the convergence and correctness of GaBP are known, but GaBP may fail to converge to the correct solution given an arbitrary positive definite covariance matrix. As was observed by Malioutov et al. (2006), the…

## Figures from this paper

## 24 Citations

Regularized Gaussian Belief Propagation with Nodes of Arbitrary Size

- Computer ScienceJ. Mach. Learn. Res.
- 2020

It is proved that, given sufficient regularization, this Gaussian belief propagation algorithm will converge and provide the exact marginal means at convergence, regardless of the way variables are assigned to nodes.

On the Convergence of Gaussian Belief Propagation with Nodes of Arbitrary Size

- Computer ScienceJ. Mach. Learn. Res.
- 2019

It is argued that GaBP-m is robust towards a certain change in variables, a property not shared by iterative solvers of linear systems, such as the conjugate gradient (CG) and preconditioned conjugATE gradient (PCG) methods.

Regularized Gaussian belief propagation

- Computer ScienceStat. Comput.
- 2018

This article investigates a regularized BP scheme by focusing on loopy Markov graphs induced by a multivariate Gaussian distribution in canonical form, and shows that the adjusted BP will always converge, with sufficient tuning, while maintaining the exact marginal means.

Convergence rate analysis of Gaussian belief propagation for Markov networks

- Computer Science, MathematicsIEEE/CAA Journal of Automatica Sinica
- 2020

This paper extends this convergence result further by showing that the convergence is exponential under the generalised diagonal dominance condition, and provides a simple bound for the convergence rate.

Convergence of Gaussian Belief Propagation Under General Pairwise Factorization: Connecting Gaussian MRF with Pairwise Linear Gaussian Model

- Computer ScienceJ. Mach. Learn. Res.
- 2019

The newly established link between Gaussian MRF and pairwise linear Gaussian model reveals an easily verifiable sufficient convergence condition in pairwise LinearGaussian model, which provides a unified criterion for assessing the convergence of Gaussian BP in multiple applications.

Probabilistic graphical models: distributed inference and learning models with small feedback vertex sets

- Computer Science
- 2014

Recursion FMP is developed, a purely distributed extension of FMP where all nodes use the same integrated message-passing protocol and the subgraph perturbation sampling algorithm is introduced, which makes use of any pre-existing tractable inference algorithm for a subgraph by perturbing this algorithm so as to yield asymptotically exact samples for the intended distribution.

Making Pairwise Binary Graphical Models Attractive

- Computer ScienceNIPS
- 2014

This work proposes a novel scheme that has better convergence properties than BP and provably provides better partition function estimates in many instances than TRBP and explores the properties of this special cover, which can be used to construct an algorithm with the desired properties.

Recursive FMP for distributed inference in Gaussian graphical models

- Computer Science2013 IEEE International Symposium on Information Theory
- 2013

Recursion FMP is proposed, a purely distributed extension of FMP, where all nodes use the same message-passing protocol and an inference problem on the entire graph is recursively reduced to those on smaller subgraphs in a distributed manner.

Tightness Results for Local Consistency Relaxations in Continuous MRFs

- Computer ScienceUAI
- 2014

It is shown that the messages of LBP can be used to calculate upper and lower bounds on the MAP value, and that these bounds coincide at convergence, yielding a natural stopping criterion which was not previously available.

A Lower Bound on the Partition Function of Attractive Graphical Models in the Continuous Case

- Computer Science, MathematicsAISTATS
- 2017

This work provides a graph cover based upper bound for continuous graphical models and uses this characterization (along with a continuous analog of a discrete correlationtype inequality) to show that the Bethe partition function also provides a lower bound on the true partition function of attractive graphical models in the continuous case.

## References

SHOWING 1-10 OF 32 REFERENCES

Fixing convergence of Gaussian belief propagation

- Computer Science2009 IEEE International Symposium on Information Theory
- 2009

This paper develops a double-loop algorithm for forcing convergence of GaBP and shows that using this construction, it is able to force convergence of Montanari's linear detection algorithm, in cases where it would originally fail.

Convergent message passing algorithms - a unifying view

- Computer ScienceUAI
- 2009

This paper presents a simple derivation of an abstract algorithm, tree-consistency bound optimization (TCBO) that is provably convergent in both its sum and max product forms, and shows that Wainwright's non-convergent sum-product algorithm for tree based variational bounds, is actually convergent with the right update order.

On the optimality of solutions of the max-product belief-propagation algorithm in arbitrary graphs

- Computer ScienceIEEE Trans. Inf. Theory
- 2001

It is shown that the assignment based on a fixed point of max-product is a "neighborhood maximum" of the posterior probabilities: the posterior probability of the max- product assignment is guaranteed to be greater than all other assignments in a particular large region around that assignment.

Norm-Product Belief Propagation: Primal-Dual Message-Passing for Approximate Inference

- Computer ScienceIEEE Transactions on Information Theory
- 2010

This paper generalizes the belief propagation algorithms of sum and max-product algorithms and introduces a new set of convergent algorithms based on “convex-free-energy” and linear-programming (LP) relaxation as a zero-temperature of a convex- free-energy.

Correctness of Belief Propagation in Gaussian Graphical Models of Arbitrary Topology

- Computer ScienceNeural Computation
- 2001

This work analyzes belief propagation in networks with arbitrary topologies when the nodes in the graph describe jointly gaussian random variables and gives an analytical formula relating the true posterior probabilities with those calculated using loopy propagation.

MAP estimation via agreement on trees: message-passing and linear programming

- Computer ScienceIEEE Transactions on Information Theory
- 2005

This work develops and analyze methods for computing provably optimal maximum a posteriori probability (MAP) configurations for a subclass of Markov random fields defined on graphs with cycles and establishes a connection between a certain LP relaxation of the mode-finding problem and a reweighted form of the max-product (min-sum) message-passing algorithm.

Tree consistency and bounds on the performance of the max-product algorithm and its generalizations

- Computer Science, MathematicsStat. Comput.
- 2004

A novel perspective on the max-product algorithm is provided, based on the idea of reparameterizing the distribution in terms of so-called pseudo-max-marginals on nodes and edges of the graph, to provide conceptual insight into the algorithm in application to graphs with cycles.

MAP estimation via agreement on (hyper)trees: Message-passing and linear programming

- Computer ScienceArXiv
- 2005

This work develops and analyzes methods for computing provably optimal MAP configurations for a subclass of Markov random fields defined on graphs with cycles, and establishes a connection between a certain LP relaxation of the mode-finding problem, and a reweighted form of the max-products message-passing algorithm.

Tree-reweighted belief propagation algorithms and approximate ML estimation by pseudo-moment matching

- Computer ScienceAISTATS
- 2003

A class of local message-passing algorithms is developed, which is called tree-reweighted belief propagation, for efficiently computing the value of these upper bounds on the log partition function of an arbitrary undirected graphical model, as well as the associated pseudomarginals.

Fixing Max-Product: Convergent Message Passing Algorithms for MAP LP-Relaxations

- Computer ScienceNIPS
- 2007

A novel message passing algorithm for approximating the MAP problem in graphical models that is derived via block coordinate descent in a dual of the LP relaxation of MAP, but does not require any tunable parameters such as step size or tree weights.