Corpus ID: 231749984

Reinforcement Learning for Decision-Making and Control in Power Systems: Tutorial, Review, and Vision

@article{Chen2021ReinforcementLF,
  title={Reinforcement Learning for Decision-Making and Control in Power Systems: Tutorial, Review, and Vision},
  author={Xin Chen and Guannan Qu and Yujie Tang and Steven H. Low and Na Li},
  journal={ArXiv},
  year={2021},
  volume={abs/2102.01168}
}
With large-scale integration of renewable generation and distributed energy resources (DERs), modern power systems are confronted with new operational challenges, such as growing complexity, increasing uncertainty, and aggravating volatility. Meanwhile, more and more data are becoming available owing to the widespread deployment of smart meters, smart sensors, and upgraded communication networks. As a result, data-driven control techniques, especially reinforcement learning (RL), have attracted… Expand

Figures and Tables from this paper

Understanding the Safety Requirements for Learning-based Power Systems Operations
TLDR
The design requirements on both data and model side of such learning algorithms by exploiting the impacts of adversarial attacks on safety critical system operations are explored, and possible measures of machine learning robustness and security are discussed for power systems operation tasks. Expand
Stability Constrained Reinforcement Learning for Real-Time Voltage Control
TLDR
This paper proposes a stability constrained reinforcement learning method for real-time voltage control in distribution grids and proves that the proposed approach provides a formal voltage stability guarantee. Expand
Bi-level Off-policy Reinforcement Learning for Volt/VAR Control Involving Continuous and Discrete Devices
  • Haotian Liu, Wenchuan Wu
  • Computer Science, Engineering
  • ArXiv
  • 2021
TLDR
A novel bi-level off-policy reinforcement learning (RL) algorithm is proposed to solve this problem in a model-free manner and support that the proposed method outperforms existing two-timescale VVC methods. Expand
Reinforcement Learning for Optimal Primary Frequency Control from Inverter-Based Resources: A Lyapunov Approach
The increase in penetration of inverter-based resources provides us with more flexibility in frequency regulation compared to conventional linear droop controllers. Using their power electronicExpand
Reinforcement Learning based Proactive Control for Transmission Grid Resilience to Wildfire
TLDR
The results show that the proposed approach can help the operator to reduce load loss during an extreme event, reduce power flow through lines that are to be de-energized, and reduce the likelihood of infeasible power-flow solutions, which would indicate violation of shortterm thermal limits of transmission lines. Expand
Lyapunov-Regularized Reinforcement Learning for Power System Transient Stability
TLDR
A Lyapunov regularized RL approach for optimal frequency control for transient stability in lossy networks is proposed and case study shows that introducing the Lyap unov regularization enables the controller to be stabilizing and achieve smaller losses. Expand
Online Optimal Control with Affine Constraints
TLDR
This paper considers online optimal control with affine constraints on the states and actions under linear dynamics with random disturbances, and proposes Online Gradient Descent with Buffer Zone (OGD-BZ), which can guarantee the system to satisfy all the constraints despite any realization of the disturbances under proper parameters. Expand
Model-Free Optimal Voltage Control via Continuous-Time Zeroth-Order Methods
  • Xin Chen, J. Poveda, Na Li
  • Computer Science, Mathematics
  • ArXiv
  • 2021
TLDR
A model-free optimal voltage control algorithm based on projected primal-dual gradient dynamics and continuous-time zeroth-order method (extreme seeking control) that can drive the voltage magnitudes back to the acceptable range and minimizes the total operating cost. Expand
Online Learning and Distributed Control for Residential Demand Response
TLDR
This paper formulate the AC control problem in a DR event as a Markov decision process that integrates the indoor thermal dynamics and customer opt-out status transition, and proposes an online DR control algorithm to learn the customer behaviors and make real-time AC control schemes. Expand
Reinforcement Learning for Optimal Frequency Control: A Lyapunov Approach
TLDR
This paper proposes to explicitly engineer the structure of neural network based controllers such that they guarantee system stability for all topologies and parameters by using a Lyapunov function to guide their structures. Expand
...
1
2
...

References

SHOWING 1-10 OF 181 REFERENCES
Reinforcement Learning and Its Applications in Modern Power and Energy Systems: A Review
TLDR
This paper provides a comprehensive literature review of RL in terms of basic ideas, various types of algorithms, and their applications in power and energy systems. Expand
Deep reinforcement learning for power system applications: An overview
TLDR
The basic ideas, models, algorithms and techniques of Deep reinforcement learning, a combination of deep learning (DL) and reinforcement learning (RL), are reviewed and applications in power systems such as energy management, demand response, electricity market, operational control and others are considered. Expand
A Data-Driven Multi-Agent Autonomous Voltage Control Framework Using Deep Reinforcement Learning
TLDR
A multi-agent AVC (MA-AVC) algorithm based on a multi- agent deep deterministic policy gradient (MADDPG) method that features centralized training and decentralized execution is developed to solve the AVC problem. Expand
(Deep) Reinforcement learning for electric power system control and related problems: A short review and perspectives
  • M. Glavic
  • Computer Science, Engineering
  • Annu. Rev. Control.
  • 2019
TLDR
The focus is only on the works published (or “in press”) in journals and books while conference publications are not included, with due attention paid to the control-related problems considerations. Expand
Deep-Reinforcement-Learning-Based Autonomous Voltage Control for Power Grid Operations
TLDR
A novel autonomous control framework “Grid Mind” is proposed for the secure operation of power grids based on cutting-edge artificial intelligence (AI) technologies that provides a data-driven, model-free and closed-loop control agent trained using deep reinforcement learning (DRL) algorithms. Expand
A Learning-Based Power Management Method for Networked Microgrids Under Incomplete Information
TLDR
Numerical experiments have verified that, compared to previous works in the literature, the proposed privacy-preserving learning model has better adaptability and enhanced computational speed. Expand
Load Frequency Control: A Deep Multi-Agent Reinforcement Learning Approach
TLDR
This paper proves using three test systems, with two, four and eight generators, that the Multi-Agent Reinforcement Learning approach can efficiently be used to perform frequency control in a decentralised way. Expand
On-Line Building Energy Optimization Using Deep Reinforcement Learning
TLDR
The benefits of using deep reinforcement learning, a hybrid type of methods that combines reinforcement learning with deep learning, to perform on-line optimization of schedules for building energy management systems, are explored for the first time in the smart grid context. Expand
Consensus Multi-Agent Reinforcement Learning for Volt-VAR Control in Power Distribution Networks
  • Yuanqi Gao, Wei Wang, N. Yu
  • Computer Science, Engineering
  • IEEE Transactions on Smart Grid
  • 2021
TLDR
A consensus multi-agent deep reinforcement learning algorithm is proposed to solve the VVC problem, which determines the operation schedules for voltage regulators, on-load tap changers, and capacitors. Expand
Scalable Voltage Control using Structure-Driven Hierarchical Deep Reinforcement Learning
TLDR
This paper uses the area-wise division structure of the power system to propose a hierarchical DRL design that can be scaled to the larger grid models, and employs an enhanced augmented random search algorithm that is tailored for the voltage control problem in a twolevel architecture. Expand
...
1
2
3
4
5
...