# Roll-back Hamiltonian Monte Carlo

@article{Yi2017RollbackHM, title={Roll-back Hamiltonian Monte Carlo}, author={Kexin Yi and Finale Doshi-Velez}, journal={arXiv: Machine Learning}, year={2017} }

We propose a new framework for Hamiltonian Monte Carlo (HMC) on truncated probability distributions with smooth underlying density functions. Traditional HMC requires computing the gradient of potential function associated with the target distribution, and therefore does not perform its full power on truncated distributions due to lack of continuity and differentiability. In our framework, we introduce a sharp sigmoid factor in the density function to approximate the probability drop at the…

## 8 Citations

### Modified Hamiltonian Monte Carlo for Bayesian inference

- Computer ScienceStat. Comput.
- 2020

It is shown that performance of HMC can be significantly improved by incorporating importance sampling and an irreversible part of the dynamics into a chain, and is called Mix & Match Hamiltonian Monte Carlo (MMHMC).

### Truncated Log-concave Sampling with Reflective Hamiltonian Monte Carlo

- Computer ScienceArXiv
- 2021

We introduce Reflective Hamiltonian Monte Carlo (ReHMC), an HMC-based algorithm, to sample from a log-concave distribution restricted to a convex polytope. We prove that, starting from a warm start,…

### On Using Hamiltonian Monte Carlo Sampling for Reinforcement Learning Problems in High-dimension

- Computer Science
- 2020

A framework, called Hamiltonian Q -Learning, is introduced that demonstrates, both theoretically and empirically, that Q values can be learned from a dataset generated by HMC samples of actions, rewards, and state transitions, and broadens the scope of RL algorithms for real-world applications.

### Hamiltonian Q-Learning: Leveraging Importance-sampling for Data Efficient RL

- Computer ScienceArXiv
- 2020

Hamiltonian Q-Learning is introduced, a data efficient modification of the Q-learning approach, which adopts an importance-sampling based technique for computing the Q function and exploits the latent low-rank structure of the dynamic system.

### PoRB-Nets: Poisson Process Radial Basis Function Networks

- Computer ScienceUAI
- 2020

A novel prior over radial basis function networks (RBFNs) is presented that allows for independent specification of functional amplitude variance and lengthscale (i.e., smoothness), where the inverse lengthscale corresponds to the concentration of radial basis functions.

### LF-PPL: A Low-Level First Order Probabilistic Programming Language for Non-Differentiable Models

- Computer ScienceAISTATS
- 2019

A new Low-level, First-order Probabilistic Programming Language~ (LF-PPL) suited for models containing a mix of continuous, discrete, and/or piecewise-continuous variables is developed, backed up by a mathematical formalism that ensures that any model expressed in this language has a density with measure zero discontinuities to maintain the validity of the inference engine.

### Markov Chain Monte Carlo Methods for Estimating Systemic Risk Allocations

- EconomicsRisks
- 2020

In this paper, we propose a novel framework for estimating systemic risk measures and risk allocations based on Markov Chain Monte Carlo (MCMC) methods. We consider a class of allocations whose jth…

### On Using Hamiltonian Monte Carlo Sampling for RL

- Computer Science
- 2022

This work introduces Hamiltonian Q -Learning and uses it to demonstrate, theoretically and empirically, that Q values can be learned from a dataset generated by HMC samples of actions, rewards, and state transitions.

## References

SHOWING 1-10 OF 18 REFERENCES

### Spherical Hamiltonian Monte Carlo for Constrained Target Distributions

- Computer ScienceICML
- 2014

A novel Markov Chain Monte Carlo (MCMC) method is proposed that provides a general and computationally efficient framework for handling boundary conditions inStatistical models with constrained probability distributions including truncated Gaussian, Bayesian Lasso, Bayes bridge regression, and a copula model for identifying synchrony among multiple neurons.

### MCMC Using Hamiltonian Dynamics

- Physics
- 2011

Hamiltonian dynamics can be used to produce distant proposals for the Metropolis algorithm, thereby avoiding the slow exploration of the state space that results from the diffusive behaviour of…

### Reflection, Refraction, and Hamiltonian Monte Carlo

- PhysicsNIPS
- 2015

A modification of the Leapfrog discretization of Hamiltonian dynamics on piecewise continuous energies, where intersections of the trajectory with discontinuities are detected, and the momentum is reflected or refracted to compensate for the change in energy.

### Auxiliary-variable Exact Hamiltonian Monte Carlo Samplers for Binary Distributions

- Computer Science, MathematicsNIPS
- 2013

We present a new approach to sample from generic binary distributions, based on an exact Hamiltonian Monte Carlo algorithm applied to a piecewise continuous augmentation of the binary distribution of…

### The No-U-turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo

- Computer ScienceJ. Mach. Learn. Res.
- 2014

The No-U-Turn Sampler (NUTS), an extension to HMC that eliminates the need to set a number of steps L, and derives a method for adapting the step size parameter {\epsilon} on the fly based on primal-dual averaging.

### Introduction to Monte Carlo Methods

- Physics
- 2009

Monte Carlo methods play an important role in scientific computation, especially when problems have a vast phase space. In this lecture an introduction to the Monte Carlo method is given. Concepts…

### A Conceptual Introduction to Hamiltonian Monte Carlo

- Medicine
- 2017

This review provides a comprehensive conceptual account of these theoretical foundations of Hamiltonian Monte Carlo, focusing on developing a principled intuition behind the method and its optimal implementations rather of any exhaustive rigor.

### Bayesian Non-negative Matrix Factorization

- Computer ScienceICA
- 2009

An iterated conditional modes algorithm is presented that rivals existing state-of-the-art NMF algorithms on an image feature extraction problem and discusses how the Gibbs sampler can be used for model order selection by estimating the marginal likelihood.

### Handbook of Markov Chain Monte Carlo

- Computer Science
- 2011

A Markov chain Monte Carlo based analysis of a multilevel model for functional MRI data and its applications in environmental epidemiology, educational research, and fisheries science are studied.

### Introduction to Monte Carlo Methods

- BiologyLearning in Graphical Models
- 1998

This chapter describes a sequence of Monte Carlo methods: importance sampling, rejection sampling, the Metropolis method, and Gibbs sampling, including methods for reducing random walk behaviour.