- Published 2002 in IEEE Transactions on Information Theory

We introduce a new class of upper bounds on the log partition function of a Markov random field (MRF). This quantity plays an important role in various contexts, including approximating marginal distributions, parameter estimation, combinatorial enumeration, statistical decision theory, and large-deviations bounds. Our derivation is based on concepts from convex duality and information geometry: in particular, it exploits mixtures of distributions in the exponential domain, and the Legendre mapping between exponential and mean parameters. In the special case of convex combinations of tree-structured distributions, we obtain a family of variational problems, similar to the Bethe variational problem, but distinguished by the following desirable properties: i) they are convex, and have a unique global optimum; and ii) the optimum gives an upper bound on the log partition function. This optimum is defined by stationary conditions very similar to those defining fixed points of the sum-product algorithm, or more generally, any local optimum of the Bethe variational problem. As with sum-product fixed points, the elements of the optimizing argument can be used as approximations to the marginals of the original model. The analysis extends naturally to convex combinations of hypertree-structured distributions, thereby establishing links to Kikuchi approximations and variants.

Citations per Year

Semantic Scholar estimates that this publication has **397** citations based on the available data.

See our **FAQ** for additional information.

Showing 1-10 of 271 extracted citations

Highly Influenced

8 Excerpts

Highly Influenced

7 Excerpts

Highly Influenced

20 Excerpts

Highly Influenced

17 Excerpts

Highly Influenced

14 Excerpts

Highly Influenced

8 Excerpts

Highly Influenced

8 Excerpts

Highly Influenced

6 Excerpts

Highly Influenced

4 Excerpts

Highly Influenced

10 Excerpts

@article{Wainwright2002ANC,
title={A new class of upper bounds on the log partition function},
author={Martin J. Wainwright and Tommi S. Jaakkola and Alan S. Willsky},
journal={IEEE Transactions on Information Theory},
year={2002},
volume={51},
pages={2313-2335}
}