Stochastic gradient method with accelerated stochastic dynamics

@article{Ohzeki2015StochasticGM,
  title={Stochastic gradient method with accelerated stochastic dynamics},
  author={Masayuki Ohzeki},
  journal={Journal of Physics: Conference Series},
  year={2015},
  volume={699}
}
  • Masayuki Ohzeki
  • Published 19 November 2015
  • Computer Science
  • Journal of Physics: Conference Series
We implement the simple method to accelerate the convergence speed to the steady state and enhance the mixing rate to the stochastic gradient Langevin method. The ordinary stochastic gradient method is based on mini-batch learning for reducing the computational cost when the amount of data is extraordinary large. The stochasticity of the gradient can be mitigated by the injection of Gaussian noise, which yields the stochastic Langevin gradient method; this method can be used for Bayesian… 
4 Citations
Efficient Irreversible Monte Carlo Samplers.
We present here two irreversible Markov chain Monte Carlo algorithms for general discrete state systems, one of the algorithms is based on the random-scan Gibbs sampler for discrete states and the
Eigenvalue analysis of an irreversible random walk with skew detailed balance conditions.
TLDR
It is found that the performance in irreversible MCMC methods violating the detailed balance condition is improved by appropriately choosing parameters in the algorithm.
Conflict between fastest relaxation of a Markov process and detailed balance condition.
TLDR
The brachistochrone method is applied to the continuous-time master equation for finite-size systems and it is found that the solution violates the detailed balance condition.
Sparse modeling for Quantum Monte-Carlo simulation
We show a new kind of applications of the sparse modeling to a traditional problem in the condensed-matter physics. In the quantum Monte-Carlo simulation, we observe a huge amount of data for

References

SHOWING 1-10 OF 50 REFERENCES
Mathematical understanding of detailed balance condition violation and its application to Langevin dynamics
TLDR
It is confirmed that the numerical implementation of the proposed method actually demonstrates two major beneficial improvements: acceleration of the relaxation to the predetermined distribution and reduction of the correlation time between two different realizations in the steady state.
Markov chain Monte Carlo method without detailed balance.
TLDR
A bounce-free worm algorithm for generic quantum spin models is formulated and it is demonstrated that the autocorrelation time of the Potts model becomes more than 6 times shorter than that by the conventional Metropolis algorithm.
Violation of detailed balance accelerates relaxation.
TLDR
It is shown that the DBC violation always makes the relaxation faster, which implies the existence of a kind of thermodynamic inequality that connects the nonequilibrium process relaxing toward steady state with the relaxation process which has the same probability distribution as its equilibrium state.
Full-order fluctuation-dissipation relation for a class of nonequilibrium steady states.
TLDR
The fluctuation-dissipation relation for nonequilibrium steady states is derived without recourse to a linear response approximation by accentuating that specific nonconservative forces relate to the large deviation of total heat in an equilibrium state.
Conflict between fastest relaxation of a Markov process and detailed balance condition.
TLDR
The brachistochrone method is applied to the continuous-time master equation for finite-size systems and it is found that the solution violates the detailed balance condition.
Annealed importance sampling
TLDR
It is shown how one can use the Markov chain transitions for such an annealing sequence to define an importance sampler, which can be seen as a generalization of a recently-proposed variant of sequential importance sampling.
L1-regularized Boltzmann machine learning using majorizer minimization
TLDR
This study utilizes the majorizer minimization method, which is a well-known technique implemented in optimization problems, to avoid the non-smoothness of the cost function.
Pseudolikelihood decimation algorithm improving the inference of the interaction network in a general class of Ising models.
TLDR
A new method to infer the topology of the interaction network in pairwise models with Ising variables is proposed that recursively sets to zero the less significant couplings, until the variation of the pseudolikelihood signals that relevant couplings are being removed.
The Bethe approximation for solving the inverse Ising problem: a comparison with other inference methods
TLDR
The formulas for several mean-field approximations are summarized and new analytical expressions for the Bethe approximation are derived, which allow one to solve the inverse Ising problem without running the susceptibility propagation algorithm (thus avoiding the lack of convergence).
...
...