A new class of upper bounds on the log partition function


We introduce a new class of upper bounds on the log partition function of a Markov random field (MRF). This quantity plays an important role in various contexts, including approximating marginal distributions, parameter estimation, combinatorial enumeration, statistical decision theory, and large-deviations bounds. Our derivation is based on concepts from convex duality and information geometry: in particular, it exploits mixtures of distributions in the exponential domain, and the Legendre mapping between exponential and mean parameters. In the special case of convex combinations of tree-structured distributions, we obtain a family of variational problems, similar to the Bethe variational problem, but distinguished by the following desirable properties: i) they are convex, and have a unique global optimum; and ii) the optimum gives an upper bound on the log partition function. This optimum is defined by stationary conditions very similar to those defining fixed points of the sum-product algorithm, or more generally, any local optimum of the Bethe variational problem. As with sum-product fixed points, the elements of the optimizing argument can be used as approximations to the marginals of the original model. The analysis extends naturally to convex combinations of hypertree-structured distributions, thereby establishing links to Kikuchi approximations and variants.

DOI: 10.1109/TIT.2005.850091

Extracted Key Phrases

14 Figures and Tables

Showing 1-10 of 38 references

Understanding belief propagation and its generalizations

  • J S Yedidia, W T Freeman, Y Weiss
  • 2002
Highly Influential
3 Excerpts

Iterative algorithms and free energy minimization

  • P Pakzad, V Anantharam
  • 2002
1 Excerpt
Citations per Year

374 Citations

Semantic Scholar estimates that this publication has received between 307 and 459 citations based on the available data.

See our FAQ for additional information.