# Adaptive Dynamic Programming: An Introduction

@article{Wang2009AdaptiveDP, title={Adaptive Dynamic Programming: An Introduction}, author={F. Wang and Huaguang Zhang and Derong Liu}, journal={IEEE Computational Intelligence Magazine}, year={2009}, volume={4} }

In this article, we introduce some recent research trends within the field of adaptive/approximate dynamic programming (ADP), including the variations on the structure of ADP schemes, the development of ADP algorithms and applications of ADP schemes. For ADP algorithms, the point of focus is that iterative algorithms of ADP can be sorted into two classes: one class is the iterative algorithm with initial stable policy; the other is the one without the requirement of initial stable policy. It is… Expand

#### 659 Citations

An Overview of Research on Adaptive Dynamic Programming

- Computer Science
- 2013

This paper gives a review of ADP in the order of the variation on the structure of ADp scheme, the development ofADP algorithms and applications, aiming to bring the reader into this novel field of optimization technology. Expand

Adaptive Dynamic Programming for Control: A Survey and Recent Advances

- Computer Science
- IEEE Transactions on Systems, Man, and Cybernetics: Systems
- 2021

Through a comprehensive and complete investigation of its applications in many existing fields, this article fully demonstrates that the ADP intelligent control method is promising in today’s artificial intelligence era. Expand

Overview of Adaptive Dynamic Programming

- Computer Science
- 2017

This chapter reviews the development of adaptive dynamic programming (ADP). It starts with a background overview of reinforcement learning and dynamic programming. It then moves on to the basic forms… Expand

A Summary on Some Typical Adaptive Dynamic Programming Schemes

- Computer Science
- 2018 37th Chinese Control Conference (CCC)
- 2018

This paper sums up four typical schemes of adaptive dynamic programming (ADP). The diagrams are provided and the algorithms of various schemes are described, which is convenient for comparison. Some… Expand

Robust adaptive dynamic programming: An overview of recent results

- Computer Science
- 2012

An application of robust-ADP to the decentralized optimal stabilization of large-scale systems is studied and an example of power systems is numerically simulated to validate the efficiency of the robust- ADP-based optimal control design. Expand

Adaptive Dynamic Programming - Discrete Version

- Computer Science
- 2018

This chapter presents the application of adaptive structures to the Bellman’s DP method to approximate the value function. Such action resulted in the creation of a family of neural dynamic… Expand

A Supplementary Condition for the Convergence of the Control Policy during Adaptive Dynamic Programming

- Mathematics
- 2018

Reinforcement learning based adaptive/approximate dynamic programming (ADP) is a powerful technique to determine an approximate optimal controller for a dynamical system. These methods bypass the… Expand

Dual iterative adaptive dynamic programming for a class of discrete-time nonlinear systems with time-delays

- Computer Science
- Neural Computing and Applications
- 2012

In this paper, a new dual iterative adaptive dynamic programming (ADP) algorithm is developed to solve optimal control problems for a class of nonlinear systems with time-delays in state and control… Expand

Adaptive dynamic programming with stable value iteration algorithm for discrete-time nonlinear systems

- Mathematics, Computer Science
- The 2012 International Joint Conference on Neural Networks (IJCNN)
- 2012

It is proved that any of iterative control obtained in the proposed algorithm can stabilize the nonlinear system which overcomes the disadvantage of traditional value iteration algorithms. Expand

A Novel Iterative $\theta $-Adaptive Dynamic Programming for Discrete-Time Nonlinear Systems

- Computer Science
- IEEE Transactions on Automation Science and Engineering
- 2014

It is proved that all the Iterative controls obtained in the iterative θ-ADP algorithm can stabilize the nonlinear system which means that the iteratives θ, which is feasible for implementations both online and offline, is feasible. Expand

#### References

SHOWING 1-10 OF 144 REFERENCES

A performance gradient perspective on approximate dynamic programming and its application to partially observable Markov decision processes

- Computer Science
- 2006 IEEE Conference on Computer Aided Control System Design, 2006 IEEE International Conference on Control Applications, 2006 IEEE International Symposium on Intelligent Control
- 2006

This paper shows an approach to integrating common approximate dynamic programming (ADP) algorithms into a theoretical framework to address both analytical characteristics and algorithmic features.… Expand

Model-free Approximate Dynamic Programming Schemes for Linear Systems

- Computer Science, Mathematics
- 2007 International Joint Conference on Neural Networks
- 2007

Online model-free adaptive critic schemes based on approximate dynamic programming (ADP) to solve optimal control problems in both discrete-time and continuous-time domains for linear systems with unknown dynamics are presented. Expand

Direct Neural Dynamic Programming

- Computer Science
- 2004

This chapter discusses the relationships, results, and challenges of various approaches under the theme of ADP, and introduces the fundamental principles of the direct neural dynamic programming (NDP), which is demonstrated for a continuous state control problem using an industrial scale Apache helicopter model. Expand

Adaptive dynamic programming

- Mathematics, Computer Science
- IEEE Trans. Syst. Man Cybern. Part C
- 2002

An adaptive dynamic programming algorithm (ADPA) is described which fuses soft computing techniques to learn the optimal cost functional for a stabilizable nonlinear system with unknown dynamics and hard Computing techniques to verify the stability and convergence of the algorithm. Expand

A Novel Infinite-Time Optimal Tracking Control Scheme for a Class of Discrete-Time Nonlinear Systems via the Greedy HDP Iteration Algorithm

- Computer Science, Medicine
- IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics)
- 2008

In this paper, we aim to solve the infinite-time optimal tracking control problem for a class of discrete-time nonlinear systems using the greedy heuristic dynamic programming (HDP) iteration… Expand

Continuous-Time ADP for Linear Systems with Partially Unknown Dynamics

- Mathematics
- 2007 IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning
- 2007

Approximate dynamic programming has been formulated and applied mainly to discrete-time systems. Expressing the ADP concept for continuous-time systems raises difficult issues related to sampling… Expand

Discrete-Time Nonlinear HJB Solution Using Approximate Dynamic Programming: Convergence Proof

- Mathematics, Computer Science
- IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics)
- 2008

It is shown that HDP converges to the optimal control and the optimal value function that solves the Hamilton-Jacobi-Bellman equation appearing in infinite-horizon discrete-time (DT) nonlinear optimal control. Expand

Neural-Network-Based Near-Optimal Control for a Class of Discrete-Time Affine Nonlinear Systems With Control Constraints

- Mathematics, Computer Science
- IEEE Transactions on Neural Networks
- 2009

In this paper, the near-optimal control problem for a class of nonlinear discrete-time systems with control constraints is solved by iterative adaptive dynamic programming algorithm. First, a novel… Expand

Handbook of Learning and Approximate Dynamic Programming

- Computer Science
- IEEE Transactions on Automatic Control
- 2006

This chapter discusses reinforcement learning in large, high-dimensional state spaces, model-based adaptive critic designs, and applications of approximate dynamic programming in power systems control. Expand

Model-free multiobjective approximate dynamic programming for discrete-time nonlinear systems with general performance index functions

- Mathematics, Computer Science
- Neurocomputing
- 2009

In this paper, a forward-in-time optimal control method for a class of discrete-time nonlinear systems with general multiobjective performance indices is proposed with unknown system dynamics. The… Expand