The Uncertainty Bellman Equation and Exploration

  title={The Uncertainty Bellman Equation and Exploration},
  author={Brendan O'Donoghue and Ian Osband and R{\'e}mi Munos and Volodymyr Mnih},
We consider the exploration/exploitation problem in reinforcement learning. For exploitation, it is well known that the Bellman equation connects the value at any time-step to the expected value at subsequent time-steps. In this paper we consider a similar uncertainty Bellman equation (UBE), which connects the uncertainty at any time-step to the expected uncertainties at subsequent time-steps, thereby extending the potential exploratory benefit of a policy beyond individual time-steps. We prove… CONTINUE READING
Highly Cited
This paper has 29 citations. REVIEW CITATIONS


Publications citing this paper.