• Corpus ID: 551162

MCMC assisted by Belief Propagaion

  title={MCMC assisted by Belief Propagaion},
  author={Sungsoo Ahn and Michael Chertkov and Jinwoo Shin},
Markov Chain Monte Carlo (MCMC) and Belief Propagation (BP) are the most popular algorithms for computational inference in Graphical Models (GM). In principle, MCMC is an exact probabilistic method which, however, often suffers from exponentially slow mixing. In contrast, BP is a deterministic method, which is typically fast, empirically very successful, however in general lacking control of accuracy over loopy graphs. In this paper, we introduce MCMC algorithms correcting the approximation… 

Figures from this paper


Approximate Inference on Planar Graphs using Loop Calculus and Belief Propagation
An algorithm is developed which represents an efficient truncation scheme on planar graphs and a new representation of the series in terms of Pfaffians of matrices, and it is shown that the first term of the Pfaffian series can provide very accurate approximations.
Belief Optimization for Binary Networks: A Stable Alternative to Loopy Belief Propagation
A novel inference algorithm for arbitrary, binary, undirected graphs that directly descend on the Bethe free energy, which is ideally suited for learning graphical models from data.
CCCP Algorithms to Minimize the Bethe and Kikuchi Free Energies: Convergent Alternatives to Belief Propagation
  • A. Yuille
  • Computer Science
    Neural Computation
  • 2002
A class of discrete iterative algorithms that are provably convergent alternatives to believe propagation (BP) and generalized belief propagation (GBP) and are pointed out that have a large range of inference and learning applications.
Belief propagation and loop series on planar graphs
It is shown that, for planar graphs, truncating the series at single-connected loops reduces, via a map reminiscent of the Fisher transformation, to evaluating the partition function of the dimer-matching model on an auxiliary planar graph.
The Complexity of Approximating a Bethe Equilibrium
  • Jinwoo Shin
  • Computer Science
    IEEE Transactions on Information Theory
  • 2014
A message-passing algorithm solving the Bethe equation in a polynomial number of operations for general binary graphical models of n variables, where the maximum degree in the underlying graph is O(logn), which is of broader interest to understand the computational complexity of the cavity method in statistical physics.
Loop series for discrete statistical models on graphs
The derivation details, logic, and motivation for the three loop calculus introduced in Chertkov and Chernyak (2006 Phys.
Iterative Decoding of Compound Codes by Probability Propagation in Graphical Models
It is pointed out that iterative decoding algorithms for various codes, including "turbo decoding" of parallel-concatenated convolutional codes, may be viewed as probability propagation in a graphical model of the code.
Complexity of Inference in Graphical Models
It is shown that low treewidth is indeed the only structural restriction of the underlying graph that can ensure tractability, and that even for the "best case" graph structure, there is no inference algorithm with complexity polynomial in thetreewidth.
Worm algorithms for classical statistical models.
It is shown that high-temperature expansions provide a basis for the novel approach to efficient Monte Carlo simulations and local, Metropolis-type schemes using this approach appear to have dynamical critical exponents close to zero.
The Bethe Partition Function of Log-supermodular Graphical Models
  • N. Ruozzi
  • Computer Science, Mathematics
  • 2012
It is demonstrated that, for any graphical model with binary variables whose potential functions are all log-supermodular, the Bethe partition function always lower bounds the true partition function.