# Probabilistic programming in Python using PyMC3

@article{Salvatier2016ProbabilisticPI,
title={Probabilistic programming in Python using PyMC3},
author={John Salvatier and Thomas V. Wiecki and Christopher J Fonnesbeck},
journal={PeerJ Comput. Sci.},
year={2016},
volume={2},
pages={e55}
}
• Published 2016
• Computer Science
• PeerJ Comput. Sci.
Probabilistic Programming allows for automatic Bayesian inference on user-defined probabilistic models. Recent advances in Markov chain Monte Carlo (MCMC) sampling allow inference on increasingly complex models. This class of MCMC, known as Hamiltonian Monte Carlo, requires gradient information which is often not readily available. PyMC3 is a new open source Probabilistic Programming framework written in Python that uses Theano to compute gradients via automatic dierentiation as well as compile… Expand
1,043 Citations
Evaluating probabilistic programming and fast variational Bayesian inference in phylogenetics
• Medicine, Computer Science
• PeerJ
• 2019
It is shown that many commonly used phylogenetic models including the general time reversible substitution model, rate heterogeneity among sites, and a range of coalescent models can be implemented using a probabilistic programming language. Expand
Evaluating probabilistic programming and fast variational Bayesian inference in phylogenetics
• Biology, Computer Science
• 2019
It is shown that many commonly used phylogenetic models including the general time reversible (GTR) substitution model, rate heterogeneity among sites, and a range of coalescent models can be implemented using a probabilistic programming language. Expand
Probabilistic programming with programmable inference
• Computer Science
• PLDI 2018
• 2018
Inference metaprogramming enables the concise expression of probabilistic models and inference algorithms across diverse elds, such as computer vision, data science, and robotics, within a single Probabilistic programming language. Expand
Pomegranate: fast and flexible probabilistic modeling in python
An overview of the design choices in pomegranate is presented, and how they have enabled complex features to be supported by simple code, making it competitive with---or outperform---other implementations of similar algorithms. Expand
ProBO: Versatile Bayesian Optimization Using Any Probabilistic Programming Language
• Computer Science
• 2019
ProBO is developed, a BO procedure that uses only standard operations common to most PPLs, and allows a user to drop in a model built with an arbitrary PPL and use it directly in BO. Expand
Hamiltonian Monte Carlo for Probabilistic Programs with Discontinuities
• Mathematics, Computer Science
• 2018
A Simple first-order Probabilistic Programming Language (SPPL) is designed that contains a sufficient set of language restrictions together with a compilation scheme that enables both the statistical and syntactic interpretation of if-else statements in the probabilistic program, within the scope of first- order PPLs. Expand
Expectation Programming
• Computer Science
• ArXiv
• 2021
A particular instantiation of the EPF concept is realized by extending the probabilistic programming language Turing to allow so-called target-aware inference to be run automatically, and it is shown that this leads to significant empirical gains compared to conventional posterior-based inference. Expand
ProBO: a Framework for Using Probabilistic Programming in Bayesian Optimization
• Computer Science, Mathematics
• ArXiv
• 2019
This paper develops ProBO, a framework for Bayesian optimization using only standard operations common to most Probabilistic programs (PPs), and introduces a model, which is term the Bayesian Product of Experts, that integrates into ProBO and can be used to combine information from multiple models implemented with different PPs. Expand
Reversible Jump Probabilistic Programming
• Computer Science
• AISTATS
• 2019
This paper presents a method for automatically deriving a Reversible Jump Markov chain Monte Carlo sampler from probabilistic programs that specify the target and proposal distributions, which relies on the interaction of several different components, including automatic differentiation, transformation inversion, and optimised code generation. Expand
Discontinuous Hamiltonian Monte Carlo for Probabilistic Programs
• Computer Science
• ArXiv
• 2018
This paper introduces a sufficient set of language restrictions, a corresponding mathematical formalism that ensures that any joint density denoted in such a language has a suitably low measure of discontinuous points, and a recipe for how to apply DHMC in the more general probabilistic-programming context. Expand

#### References

SHOWING 1-10 OF 35 REFERENCES
A New Approach to Probabilistic Programming Inference
• Mathematics, Computer Science
• AISTATS
• 2014
A new approach to inference in expressive probabilistic programming languages based on particle Markov chain Monte Carlo that supports accurate inference in models that make use of complex control ow, including stochastic recursion is introduced. Expand
Venture: a higher-order probabilistic programming platform with programmable inference
• Computer Science, Mathematics
• ArXiv
• 2014
Stochastic regeneration linear runtime scaling in cases where many previous approaches scaled quadratically is shown, and how to use stochastic regeneration and the SPI to implement general-purpose inference strategies such as Metropolis-Hastings, Gibbs sampling, and blocked proposals based on particle Markov chain Monte Carlo and mean-field variational inference techniques are shown. Expand
Church: a language for generative models
• Computer Science
• UAI
• 2008
This work introduces Church, a universal language for describing stochastic generative processes, based on the Lisp model of lambda calculus, containing a pure Lisp as its deterministic subset. Expand
Automatic Variational Inference in Stan
• Computer Science, Mathematics
• NIPS
• 2015
An automatic variational inference algorithm, automatic differentiation Variational inference (ADVI), which is implemented in Stan, a probabilistic programming system and can be used on any model the authors write in Stan. Expand
Frequentism and Bayesianism: A Python-driven Primer
This paper presents a brief, semi-technical comparison of the essential features of the frequentist and Bayesian approaches to statistical inference, with several illustrative examples implemented inExpand
Picture: A probabilistic programming language for scene perception
• Computer Science
• 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
• 2015
Picture is presented, a probabilistic programming language for scene understanding that allows researchers to express complex generative vision models, while automatically solving them using fast general-purpose inference machinery. Expand
The No-U-turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo
• Mathematics, Computer Science
• J. Mach. Learn. Res.
• 2014
The No-U-Turn Sampler (NUTS), an extension to HMC that eliminates the need to set a number of steps L, and derives a method for adapting the step size parameter {\epsilon} on the fly based on primal-dual averaging. Expand
Python reference manual
This reference manual describes the syntax and core semantics'' of the Python language, which is terse, but attempts to be exact and complete. Expand
Cython: The Best of Both Worlds
• Computer Science
• Computing in Science & Engineering
• 2011
Cython is a Python language extension that allows explicit type declarations and is compiled directly to C. As such, it addresses Python's large overhead for numerical loops and the difficulty ofExpand
The NumPy Array: A Structure for Efficient Numerical Computation
• Physics, Computer Science
• Computing in Science & Engineering
• 2011
This effort shows, NumPy performance can be improved through three techniques: vectorizing calculations, avoiding copying data in memory, and minimizing operation counts. Expand