Bridging the Gap between Stochastic Gradient MCMC and Stochastic Optimization

@article{Chen2015BridgingTG,
  title={Bridging the Gap between Stochastic Gradient MCMC and Stochastic Optimization},
  author={Changyou Chen and David E. Carlson and Zhe Gan and Chunyuan Li and Lawrence Carin},
  journal={CoRR},
  year={2015},
  volume={abs/1512.07962}
}
Stochastic gradient Markov chain Monte Carlo (SG-MCMC) methods are Bayesian analogs to popular stochastic optimization methods; however, this connection is not well studied. We explore this relationship by applying simulated annealing to an SGMCMC algorithm. Furthermore, we extend recent SG-MCMC methods with two key components: i) adaptive preconditioners (as in ADAgrad or RMSprop), and ii) adaptive element-wise momentum weights. The zerotemperature limit gives a novel stochastic optimization… CONTINUE READING
Recent Discussions
This paper has been referenced on Twitter 28 times over the past 90 days. VIEW TWEETS