Residual Algorithms: Reinforcement Learning with Function Approximation

@inproceedings{Baird1995ResidualAR,
  title={Residual Algorithms: Reinforcement Learning with Function Approximation},
  author={Leemon C. Baird},
  booktitle={ICML},
  year={1995}
}
ABSTRACT A number of reinforcement learning algorithms have been developed that are guaranteed to converge to the optimal solution when used with lookup tables. It is shown, however, that these algorithms can easily become unstable when implemented directly with a general function-approximation system, such as a sigmoidal multilayer perceptron, a radial-basis-function system, a memory-based learning system, or even a linear function-approximation system. A new class of algorithms, residual… CONTINUE READING

Citations

Publications citing this paper.
SHOWING 1-10 OF 600 CITATIONS

Multiscale Q-learning with linear function approximation

  • Discrete Event Dynamic Systems
  • 2016
VIEW 5 EXCERPTS
CITES BACKGROUND
HIGHLY INFLUENCED

Intelligent Inventory Control: Is Bootstrapping Worth Implementing?

  • Intelligent Information Processing
  • 2012
VIEW 10 EXCERPTS
CITES METHODS & BACKGROUND
HIGHLY INFLUENCED

University of Alberta Faster Gradient-td Algorithms

VIEW 4 EXCERPTS
CITES BACKGROUND & METHODS
HIGHLY INFLUENCED

Gradient temporal-difference learning algorithms

VIEW 12 EXCERPTS
CITES METHODS & BACKGROUND
HIGHLY INFLUENCED

Residual Reinforcement Learning using Neural Networks

VIEW 8 EXCERPTS
CITES BACKGROUND & METHODS
HIGHLY INFLUENCED

A Kernel Loss for Solving the Bellman Equation

  • NeurIPS 2019
  • 2019
VIEW 8 EXCERPTS
CITES BACKGROUND & METHODS
HIGHLY INFLUENCED

An Adaptive Sampling Algorithm for Policy Evaluation

  • 2019 Fifth Indian Control Conference (ICC)
  • 2019
VIEW 9 EXCERPTS
HIGHLY INFLUENCED

Bellman Residual Algorithms with Predictive Error Guarantees

VIEW 10 EXCERPTS
CITES BACKGROUND & METHODS
HIGHLY INFLUENCED

Deep Residual Reinforcement Learning

VIEW 20 EXCERPTS
CITES BACKGROUND & METHODS
HIGHLY INFLUENCED

FILTER CITATIONS BY YEAR

1995
2019

CITATION STATISTICS

  • 104 Highly Influenced Citations

  • Averaged 42 Citations per year from 2017 through 2019

  • 7% Increase in citations per year in 2019 over 2018

References

Publications referenced by this paper.
SHOWING 1-10 OF 11 REFERENCES