TD-∆ π : A Model-Free Algorithm for Efficient Exploration

@inproceedings{Silva2012TD,
title={TD-∆ π : A Model-Free Algorithm for Efficient Exploration},
author={Bruno Castro da Silva and Andrew G. Barto},
year={2012}
}

We study the problem of finding efficient exploration policies for the case in which an agent is momentarily not concerned with exploiting, and instead tries to compute a policy for later use. We first formally define the Optimal Exploration Problem as one of sequential sampling and show that its solutions correspond to paths of minimum expected length in the space of policies. We derive a model-free, local linear approximation to such solutions and use it to construct efficient exploration… CONTINUE READING