Convergent Tree-Reweighted Message Passing for Energy Minimization

  title={Convergent Tree-Reweighted Message Passing for Energy Minimization},
  author={Vladimir Kolmogorov},
  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
  • V. Kolmogorov
  • Published 1 October 2006
  • Computer Science
  • IEEE Transactions on Pattern Analysis and Machine Intelligence
Algorithms for discrete energy minimization are of fundamental importance in computer vision. In this paper, we focus on the recent technique proposed by Wainwright et al. (Nov. 2005)- tree-reweighted max-product message passing (TRW). It was inspired by the problem of maximizing a lower bound on the energy. However, the algorithm is not guaranteed to increase this bound - it may actually go down. In addition, TRW does not always converge. We develop a modification of this algorithm which we… 

Figures from this paper

Comparison of Energy Minimization Algorithms for Highly Connected Graphs
This paper proposes the problem of stereo with occlusions as a new test bed for minimization algorithms and shows that sophisticated implementations of BP and TRW have the same time and memory complexity as that of 4-connected grid-graph stereo.
Convergent message passing algorithms - a unifying view
This paper presents a simple derivation of an abstract algorithm, tree-consistency bound optimization (TCBO) that is provably convergent in both its sum and max product forms, and shows that Wainwright's non-convergent sum-product algorithm for tree based variational bounds, is actually convergent with the right update order.
Message-Passing Algorithms: Reparameterizations and Splittings
This paper provides a systematic study of message-passing algorithms that extends the known results by exhibiting new sufficient conditions for convergence to local and/or global optima, providing a combinatorial characterization of these optima based on graph covers, and describing a new convergent and correct message-Passing algorithm whose derivation unifies many of the known convergent message- passing algorithms.
Alphabet SOUP: A framework for approximate energy minimization
A general framework for finding approximate MAP solutions of arbitrary energy functions by iteratively solving subproblems over a reduced state-space and provides a theoretical guarantee on the quality of the solution when the inner loop of the algorithm is solved exactly.
Convex Optimization for Parallel Energy Minimization
This work reformulates the quadratic energy minimization problem as a total variation denoising problem, which, when viewed geometrically, enables the use of projection and reflection based convex methods and performs an extensive empirical evaluation comparing state-of-the-art combinatorial algorithms and convex optimization techniques.
Hardware-efficient belief propagation
A low-power VLSI circuit for disparity estimation that can construct 440 M messages per second and generate high quality disparity maps in near real-time and an O(L) message construction algorithm for the robust functions commonly used for describing the smoothness terms in the energy function.
Efficiently solving convex relaxations for MAP estimation
This work builds on the tree reweighted message passing (TRW) framework and shows how the dual formulation of TRW can be extended to include cycle inequalities and SOC constraints, and proposes efficient iterative algorithms for solving the resulting duals.
A Comparative Study of Energy Minimization Methods for Markov Random Fields with Smoothness-Based Priors
A set of energy minimization benchmarks are described and used to compare the solution quality and runtime of several common energy minimizations algorithms and a general-purpose software interface is provided that allows vision researchers to easily switch between optimization methods.
General Search Algorithms for Energy Minimization Problems
A scheme for solving Energy Minimization problems, which is based on the A * algorithm accomplished with appropriately chosen LP-relaxations as heuristic functions, that can be used as a stop criterion for LP based iterative algorithms.
MRF Energy Minimization & Beyond via Dual Decomposition
It is shown that, by appropriately choosing what subproblems to use, one can design novel and very powerful MRF optimization algorithms, able to derive algorithms that generalize and extend state-of-the-art message-passing methods and take full advantage of the special structure that may exist in particular MRFs.


An experimental comparison of min-cut/max- flow algorithms for energy minimization in vision
The goal of this paper is to provide an experimental comparison of the efficiency of min-cut/max flow algorithms for applications in vision, comparing the running times of several standard algorithms, as well as a new algorithm that is recently developed.
On the Optimality of Tree-reweighted Max-product Message-passing
This paper demonstrates how it is possible to identify part of the optimal solution—i.e., a provably optimal solution for a subset of nodes— without knowing a complete solution, and establishes that for submodular functions, a WTA fixed point always yields a globally optimal solution.
Efficient Belief Propagation for Early Vision
Algorithmic techniques are presented that substantially improve the running time of the loopy belief propagation approach and reduce the complexity of the inference algorithm to be linear rather than quadratic in the number of possible labels for each pixel, which is important for problems such as image restoration that have a large label set.
Globally optimal solutions for energy minimization in stereo vision using reweighted belief propagation
It is shown that for standard benchmark stereo pairs, the global optimum can be found in about 30 minutes using a variant of the belief propagation (BP) algorithm.
Fast approximate energy minimization via graph cuts
This paper proposes two algorithms that use graph cuts to compute a local minimum even when very large moves are allowed, and generates a labeling such that there is no expansion move that decreases the energy.
Tree-reweighted belief propagation algorithms and approximate ML estimation by pseudo-moment matching
A class of local message-passing algorithms is developed, which is called tree-reweighted belief propagation, for efficiently computing the value of these upper bounds on the log partition function of an arbitrary undirected graphical model, as well as the associated pseudomarginals.
MAP estimation via agreement on trees: message-passing and linear programming
This work develops and analyze methods for computing provably optimal maximum a posteriori probability (MAP) configurations for a subclass of Markov random fields defined on graphs with cycles and establishes a connection between a certain LP relaxation of the mode-finding problem and a reweighted form of the max-product (min-sum) message-passing algorithm.
Multi-camera Scene Reconstruction via Graph Cuts
This paper addresses the problem of computing the 3-dimensional shape of an arbitrary scene from a set of images taken at known viewpoints by giving an energy minimization formulation of the multi-camera scene reconstruction problem.
Constructing free-energy approximations and generalized belief propagation algorithms
This work explains how to obtain region-based free energy approximations that improve the Bethe approximation, and corresponding generalized belief propagation (GBP) algorithms, and describes empirical results showing that GBP can significantly outperform BP.
Approximation algorithms for the metric labeling problem via a new linear programming formulation
A natural integer programming formulation is introduced and it is shown that the integrality gap of its linear relaxation either matches or improves the ratios known for several cases of the metric labeling problem studied until now, providing a unified approach to solving them.