Deep reinforcement learning for turbulent drag reduction in channel flows

@article{Guastoni2023DeepRL,
  title={Deep reinforcement learning for turbulent drag reduction in channel flows},
  author={Luca Guastoni and Jean Rabault and Philipp Schlatter and Hossein Azizpour and Ricardo Vinuesa},
  journal={The European Physical Journal. E, Soft Matter},
  year={2023},
  volume={46}
}
We introduce a reinforcement learning (RL) environment to design and benchmark control strategies aimed at reducing drag in turbulent fluid flows enclosed in a channel. The environment provides a framework for computationally efficient, parallelized, high-fidelity fluid simulations, ready to interface with established RL agent programming interfaces. This allows for both testing existing deep reinforcement learning (DRL) algorithms against a challenging task, and advancing our knowledge of a… 
View on Springer
link.springer.com
10 Citations
Background Citations
4
Methods Citations
1

10 Citations

Turbulence control in plane Couette flow using low-dimensional neural ODE-based models and deep reinforcement learning

A data-driven low-dimensional DManD model of the dynamics of turbulent flows is generated, using an autoencoder and neural ordinary differential equation as the environment, and an RL control agent is trained, yielding a 440-fold speedup over training on the DNS, with equivalent control performance.

Recent advances in applying deep reinforcement learning for flow control: perspectives and future directions

This review focuses on the application of DRL to AFC, highlighting the current limitations of the DRL algorithms, and suggesting some of the potential upcoming milestones to reach, as well as open questions that are likely to attract the attention of the fluid-mechanics community.

Log-law recovery through reinforcement-learning wall model for large eddy simulation

This paper focuses on the use of reinforcement learning (RL) as a machine-learning (ML) modeling tool for near-wall turbulence. RL has demonstrated its effectiveness in solving high-dimensional…

Effective control of two-dimensional Rayleigh-Bénard convection: invariant multi-agent reinforcement learning is all you need

This work shows the potential of MARL DRL for controlling large RBC systems, as well as demonstrates the possibility for DRL to discover strategies that move the RBC configuration between different topological configurations, yielding desirable heat-transfer characteristics.

How to Control Hydrodynamic Force on Fluidic Pinball via Deep Reinforcement Learning

Deep reinforcement learning (DRL) for fluidic pinball, three individually rotating cylinders in the uniform flow arranged in an equilaterally triangular configuration, can learn the efficient flow…

Explaining wall-bounded turbulence through deep learning

Despite its great scientific and technological importance, wall-bounded turbulence is an unresolved problem that requires new perspectives to be tackled. One of the key strategies has been to study…

Learning a model is paramount for sample efficiency in reinforcement learning control of PDEs

It is demonstrated that learning an actuated model in parallel to training the RL agent significantly reduces the total amount of required data sampled from the real system, and that iteratively updating the model is of major importance to avoid biases in the RL training.

The transformative potential of machine learning for experiments in fluid mechanics

This perspective will highlight several aspects of experimental fluid mechanics that stand to benefit from progress advances in machine learning, including augmenting the fidelity and quality of measurement techniques, improving experimental design and surrogate digital-twin models and enabling real-time estimation and control.

Predicting the wall-shear stress and wall pressure through convolutional neural networks

The objective of this study is to assess the capability of convolution-based neural networks to predict wall quantities in a turbulent open channel flow. The first tests are performed by training a…

Emerging Trends in Machine Learning for Computational Fluid Dynamics

Trends in ML that are providing opportunities to advance the field of computational fluid dynamics (CFD) are focused on and synergies between ML and CFD that have already shown benefits are discussed.

70 References

Applying deep reinforcement learning to active flow control in weakly turbulent conditions

Machine learning has recently become a promising technique in fluid mechanics, especially for active flow control (AFC) applications. A recent work [J. Fluid Mech. (2019), vol. 865, pp. 281-302] has…

Deep Reinforcement Learning for Flow Control Exploits Different Physics for Increasing Reynolds Number Regimes

The increase in emissions associated with aviation requires deeper research into novel sensing and flow-control strategies to obtain improved aerodynamic performances. In this context, data-driven…

Robust active flow control over a range of Reynolds numbers using an artificial neural network trained through deep reinforcement learning

It is shown that the DRL controller is able to significantly reduce the lift and drag fluctuations and to actively reduce the drag by approximately 5.7%, at $Re$=100, 200, 300, and 400 respectively.

Accelerating deep reinforcement learning strategies of flow control through a multi-environment approach

It is shown that the problem is embarrassingly parallel and that perfect speedup can be obtained up to the batch size of the problem, with weaker scaling still taking place for a larger number of simulations.

Reinforcement learning for bluff body active flow control in experiments and simulations

Reinforcement learning is demonstrated to be effectiveness in experimental fluid mechanics and verifies it by simulations, potentially paving the way for efficient exploration of additional active flow control strategies in other complex fluid mechanics applications.

Automating Turbulence Modeling by Multi-Agent Reinforcement Learning

This work introduces multi-agent reinforcement learning (MARL) as an automated discovery tool of turbulence models and demonstrates the potential of this approach on Large Eddy Simulations of homogeneous and isotropic turbulence using as reward the recovery of the statistical properties of Direct Numerical Simulations.

Deep reinforcement learning in fluid mechanics: A promising method for both active flow control and shape optimization

An insight into the current state of the art of the use of DRL within fluid mechanics, focusing on control and optimal design problems.

Robust flow control and optimal sensor placement using deep reinforcement learning

The proposed sparsity-seeking algorithm has achieved a successful optimisation to a reduced five-sensor layout while keeping state-of-the-art performance and a novel algorithm (S-PPO-CMA) is introduced to optimise the sensor layout.

Flow Control in Wings and Discovery of Novel Approaches via Deep Reinforcement Learning

In this review, we summarize existing trends of flow control used to improve the aerodynamic efficiency of wings. We first discuss active methods to control turbulence, starting with flat-plate…

Unsupervised Deep Learning of Incompressible Fluid Dynamics

This work proposes an unsupervised framework that allows powerful deep neural networks to learn the dynamics of incompressible fluids end to end on a grid-based representation and introduces a loss function that penalizes residuals of the incompressable Navier Stokes equations.
...