Corpus ID: 195069379

Adaptive Temporal-Difference Learning for Policy Evaluation with Per-State Uncertainty Estimates

@article{Penedones2019AdaptiveTL,
  title={Adaptive Temporal-Difference Learning for Policy Evaluation with Per-State Uncertainty Estimates},
  author={Hugo Penedones and C. Riquelme and D. Vincent and Hartmut Maennel and T. Mann and Andr{\'e} Barreto and S. Gelly and Gergely Neu},
  journal={ArXiv},
  year={2019},
  volume={abs/1906.07987}
}

Hugo Penedones, C. Riquelme, +5 authors Gergely Neu

Published 2019

Computer Science, Mathematics

ArXiv

We consider the core reinforcement-learning problem of on-policy value function approximation from a batch of trajectory data, and focus on various issues of Temporal Difference (TD) learning and Monte Carlo (MC) policy evaluation. The two methods are known to achieve complementary bias-variance trade-off properties, with TD tending to achieve lower variance but potentially higher bias. In this paper, we argue that the larger bias of TD can be a result of the amplification of local… CONTINUE READING

View PDF on arXiv

Save to Library

Create Alert

Cite

Launch Research Feed