• Corpus ID: 771841

Adapting Bias by Gradient Descent: An Incremental Version of Delta-Bar-Delta

@inproceedings{Sutton1992AdaptingBB,
  title={Adapting Bias by Gradient Descent: An Incremental Version of Delta-Bar-Delta},
  author={Richard S. Sutton},
  booktitle={AAAI Conference on Artificial Intelligence},
  year={1992}
}

• R. Sutton
• Published in
  AAAI Conference on Artificial…
  12 July 1992
• Computer Science

Appropriate bias is widely viewed as the key to efficient learning and generalization. I present a new algorithm, the Incremental Delta-Bar-Delta (IDBD) algorithm, for the learning of appropriate biases based on previous learning experience. The IDBD algorithm is developed for the case of a simple, linear learning system--the LMS or delta rule with a separate learning-rate parameter for each input. The IDBD algorithm adjusts the learning-rate parameters, which are an important form of bias for this system. Because bias in this approach is adapted based on previous learning experience, the appropriate test beds are drifting or non-stationary learning tasks. For particular tasks of this type, I show that the IDBD algorithm performs better than ordinary LMS and in fact finds the optimal learning rates. The IDBD algorithm extends and improves over prior work by Jacobs and by me in that it is fully incremental and has only a single free parameter. This paper also extends previous work by presenting a derivation of the IDBD algorithm as gradient descent in the space of learning-rate parameters. Finally, I offer a novel interpretation of the IDBD algorithm as an incremental form of hold-one-out cross validation.