A new class of upper bounds on the log partition function

Abstract

We introduce a new class of upper bounds on the log partition function of a Markov random field (MRF). This quantity plays an important role in various contexts, including approximating marginal distributions, parameter estimation, combinatorial enumeration, statistical decision theory, and large-deviations bounds. Our derivation is based on concepts from convex duality and information geometry: in particular, it exploits mixtures of distributions in the exponential domain, and the Legendre mapping between exponential and mean parameters. In the special case of convex combinations of tree-structured distributions, we obtain a family of variational problems, similar to the Bethe variational problem, but distinguished by the following desirable properties: i) they are convex, and have a unique global optimum; and ii) the optimum gives an upper bound on the log partition function. This optimum is defined by stationary conditions very similar to those defining fixed points of the sum-product algorithm, or more generally, any local optimum of the Bethe variational problem. As with sum-product fixed points, the elements of the optimizing argument can be used as approximations to the marginals of the original model. The analysis extends naturally to convex combinations of hypertree-structured distributions, thereby establishing links to Kikuchi approximations and variants.

DOI: 10.1109/TIT.2005.850091

Extracted Key Phrases

14 Figures and Tables

Showing 1-10 of 38 references

Understanding belief propagation and its generalizations

  • J S Yedidia, W T Freeman, Y Weiss
  • 2002
Highly Influential
3 Excerpts

Iterative algorithms and free energy minimization

  • P Pakzad, V Anantharam
  • 2002
1 Excerpt
02040'03'05'07'09'11'13'15'17
Citations per Year

374 Citations

Semantic Scholar estimates that this publication has received between 307 and 459 citations based on the available data.

See our FAQ for additional information.