Concentration Bounds for Two Timescale Stochastic Approximation with Applications to Reinforcement Learning

@article{Dalal2017ConcentrationBF,
  title={Concentration Bounds for Two Timescale Stochastic Approximation with Applications to Reinforcement Learning},
  author={Gal Dalal and Bal{\'a}zs Sz{\"o}r{\'e}nyi and Gugan Thoppe and Shie Mannor},
  journal={CoRR},
  year={2017},
  volume={abs/1703.05376}
}
Two-timescale Stochastic Approximation (SA) algorithms are widely used in Reinforcement Learning (RL). In such methods, the iterates consist of two parts that are updated using different stepsizes. We develop the first convergence rate result for these algorithms; in particular, we provide a general methodology for analyzing two-timescale linear SA. We apply our methodology to two-timescale RL algorithms such as GTD(0), GTD2, and TDC. 
Related Discussions
This paper has been referenced on Twitter 23 times. VIEW TWEETS

From This Paper

Topics from this paper.

References

Publications referenced by this paper.
Showing 1-10 of 13 references

A concentration bound for stochastic approximation via alekseev’s formula

Thoppe, Gugan, Borkar, S Vivek
2015

Stochastic approximation: a dynamical systems viewpoint

Borkar, S Vivek
2008

The ode method for convergence of stochastic approximation and reinforcement learning

Borkar, S Vivek, Meyn, P Sean
SIAM Journal on Control and Optimization, • 2000

Similar Papers

Loading similar papers…