Corpus ID: 12870110

Fast Parallel SAME Gibbs Sampling on General Discrete Bayesian Networks

@article{Seita2015FastPS,
  title={Fast Parallel SAME Gibbs Sampling on General Discrete Bayesian Networks},
  author={Daniel Seita and Haoyu Chen and John F. Canny},
  journal={ArXiv},
  year={2015},
  volume={abs/1511.06416}
}
A fundamental task in machine learning and related fields is to perform inference on Bayesian networks. Since exact inference takes exponential time in general, a variety of approximate methods are used. Gibbs sampling is one of the most accurate approaches and provides unbiased samples from the posterior but it has historically been too expensive for large models. In this paper, we present an optimized, parallel Gibbs sampler augmented with state replication (SAME or State Augmented Marginal… Expand
3 Citations

Paper Mentions

Blog Post
GPU-accelerated Gibbs sampling: a case study of the Horseshoe Probit model
TLDR
This paper shows how to do Gibbs sampling in a fully data-parallel manner on a graphics processing unit, for a large class of exchangeable models that admit latent variable representations and finds that the implementation scales effectively to thousands of predictors and millions of data points simultaneously. Expand
GPU-accelerated Gibbs Sampling
TLDR
This paper shows how to do Gibbs sampling in a fully data-parallel manner on a graphics processing unit (GPU) for a large class of exchangeable models that admit latent variable representations and finds that the implementation scales effectively to thousands of predictors and millions of data points simultaneously. Expand
Using Deep Neural Network Approximate Bayesian Network
TLDR
A new method to approximate posterior probabilities of Bayesian Network using Deep Neural Network is presented, which is much faster and gains higher accuracy in medium sized Bayesian network. Expand

References

SHOWING 1-10 OF 24 REFERENCES
SAME but Different: Fast and High Quality Gibbs Parameter Estimation
TLDR
It is shown that combining SAME with factored sample representation (or approximation) gives throughput competitive with the fastest symbolic methods, but with potentially better quality, than other LDA implementations. Expand
Analyzing Hogwild Parallel Gaussian Gibbs Sampling
TLDR
It is shown that if the Gaussian precision matrix is generalized diagonally dominant, then any Hogwild Gibbs sampler, with any update schedule or allocation of variables to processors, yields a stable sampling process with the correct sample mean. Expand
Parallel Gibbs Sampling: From Colored Fields to Thin Junction Trees
TLDR
This work proposes two methods to construct parallel Gibbs samplers guaranteed to draw from the targeted distribution, called the Chromatic sampler and the Splash sampler, a complementary strategy which can be used when the variables are tightly coupled. Expand
Reducing the sampling complexity of topic models
TLDR
An algorithm which scales linearly with the number of actually instantiated topics kd in the document, for large document collections and in structured hierarchical models kd ll k, yields an order of magnitude speedup. Expand
Towards high-throughput gibbs sampling at scale: a study across storage managers
TLDR
This work studies how (and whether) traditional data processing choices about materialization, page layout, and buffer-replacement policy need to be changed to achieve high-throughput Gibbs sampling for factor graphs that are larger than main memory. Expand
Marginal maximum a posteriori estimation using Markov chain Monte Carlo
TLDR
A simple and novel MCMC strategy, called State-Augmentation for Marginal Estimation (SAME), which leads to MMAP estimates for Bayesian models and illustrates the simplicity and utility of the approach for missing data interpolation in autoregressive time series and blind deconvolution of impulsive processes. Expand
Graphical Models, Exponential Families, and Variational Inference
TLDR
The variational approach provides a complementary alternative to Markov chain Monte Carlo as a general source of approximation methods for inference in large-scale statistical models. Expand
A Monte Carlo Implementation of the EM Algorithm and the Poor Man's Data Augmentation Algorithms
Abstract The first part of this article presents the Monte Carlo implementation of the E step of the EM algorithm. Given the current guess to the maximizer of the posterior distribution, latent dataExpand
Accelerated Quantification of Bayesian Networks with Incomplete Data
TLDR
This paper considers statistical batch learning of the probability tables on the basis of incomplete data and expert knowledge and proposes a new class of models that allows a great variety of local functional restrictions to be imposed on the statistical model. Expand
Probabilistic Graphical Models: Principles and Techniques - Adaptive Computation and Machine Learning
TLDR
Because uncertainty is an inescapable aspect of most real-world applications, the book focuses on probabilistic models, which make the uncertainty explicit and provide models that are more faithful to reality. Expand
...
1
2
3
...