Corpus ID: 234340217

How To Train Your Program

  title={How To Train Your Program},
  author={David Tolpin},
We present a Bayesian approach to machine learning with probabilistic programs. In our approach, training on available data is implemented as inference on a hierarchical model. The posterior distribution of model parameters is then used to stochastically condition a complementary model, such that inference on new data yields the same posterior distribution of latent parameters corresponding to the new data as inference on a hierachical model on the combination of both previously available and… Expand

Figures from this paper


Learning Programs: A Hierarchical Bayesian Approach
A nonparametric hierarchical Bayesian prior over programs which shares statistical strength across multiple tasks is introduced and an MCMC algorithm is provided that can perform safe program transformations on this representation to reveal shared inter-program substructures. Expand
Inducing Probabilistic Programs by Bayesian Program Merging
This report outlines an approach to learning generative models from data that expresses models as probabilistic programs, which allows them to capture abstract patterns within the examples, and considers two types of transformation: abstraction and Deargumentation, which simplifies functions by reducing the number of arguments. Expand
Inference Compilation and Universal Probabilistic Programming
We introduce a method for using deep neural networks to amortize the cost of inference in models from the family induced by universal probabilistic programming languages, establishing a frameworkExpand
Just-In-Time Learning for Fast and Flexible Inference
This work introduces just-in-time learning, a framework for fast and flexible inference that learns to speed up inference at run-time and shows how this framework can allow us to combine the flexibility of sampling with the efficiency of deterministic message-passing. Expand
On Markov chain Monte Carlo methods for tall data
An original subsampling-based approach is proposed which samples from a distribution provably close to the posterior distribution of interest, yet can require less than $O(n)$ data point likelihood evaluations at each iteration for certain statistical models in favourable scenarios. Expand
Etalumis: bringing probabilistic programming to scientific simulators at scale
A novel PPL framework that couples directly to existing scientific simulators through a cross-platform probabilistic execution protocol and provides Markov chain Monte Carlo (MCMC) and deep-learning-based inference compilation (IC) engines for tractable inference is presented. Expand
Firefly Monte Carlo: Exact MCMC with Subsets of Data
Fly Monte Carlo (FlyMC) MCMC algorithm with auxiliary variables that only queries the likelihoods of a subset of the data at each iteration yet simulates from the exact posterior distribution, allowing MCMC methods to tackle larger datasets than were previously considered feasible. Expand
Deployable probabilistic programming
This work introduces Infergo, a probabilistic programming facility for Go, a modern programming language of choice for server-side software development, and argues that a similar probabilism programming facility can be added to most modern general-purpose programming languages. Expand
Towards scaling up Markov chain Monte Carlo: an adaptive subsampling approach
This paper describes a methodology that aims to scale up the Metropolis-Hastings (MH) algorithm by proposing an approximate implementation of the accept/reject step of MH that only requires evaluating the likelihood of a random subset of the data, yet is guaranteed to coincide with the accepted step based on the full dataset with a probability superior to a user-specified tolerance level. Expand
Austerity in MCMC Land: Cutting the Metropolis-Hastings Budget
This work introduces an approximate MH rule based on a sequential hypothesis test that allows us to accept or reject samples with high confidence using only a fraction of the data required for the exact MH rule. Expand