A new class of upper bounds on the log partition function

Abstract

We introduce a new class of upper bounds on the log partition function of a Markov random field (MRF). This quantity plays an important role in various contexts, including approximating marginal distributions, parameter estimation, combinatorial enumeration, statistical decision theory, and large-deviations bounds. Our derivation is based on concepts from convex duality and information geometry: in particular, it exploits mixtures of distributions in the exponential domain, and the Legendre mapping between exponential and mean parameters. In the special case of convex combinations of tree-structured distributions, we obtain a family of variational problems, similar to the Bethe variational problem, but distinguished by the following desirable properties: i) they are convex, and have a unique global optimum; and ii) the optimum gives an upper bound on the log partition function. This optimum is defined by stationary conditions very similar to those defining fixed points of the sum-product algorithm, or more generally, any local optimum of the Bethe variational problem. As with sum-product fixed points, the elements of the optimizing argument can be used as approximations to the marginals of the original model. The analysis extends naturally to convex combinations of hypertree-structured distributions, thereby establishing links to Kikuchi approximations and variants.

DOI: 10.1109/TIT.2005.850091

Extracted Key Phrases

14 Figures and Tables

02040'03'05'07'09'11'13'15'17
Citations per Year

397 Citations

Semantic Scholar estimates that this publication has 397 citations based on the available data.

See our FAQ for additional information.

Cite this paper

@article{Wainwright2002ANC, title={A new class of upper bounds on the log partition function}, author={Martin J. Wainwright and Tommi S. Jaakkola and Alan S. Willsky}, journal={IEEE Transactions on Information Theory}, year={2002}, volume={51}, pages={2313-2335} }