Learning to swim in potential flow

  title={Learning to swim in potential flow},
  author={Yusheng Jiao and Feng Ling and Sina Heydari and Nicolas Manfred Otto Heess and Josh Merel and Eva Kanso},
Fish swim by undulating their bodies. These propulsive motions require coordinated shape changes of a body that interacts with its fluid environment, but the specific shape coordination that leads to robust turning and swimming motions remains unclear. We propose a simple model of a three-link fish swimming in a potential flow environment and we use model-free reinforcement learning to arrive at optimal shape changes for two swimming tasks: swimming in a desired direction and swimming towards a… 

Figures from this paper

Point-to-Point Navigation of a Fish-Like Swimmer in a Vortical Flow With Deep Reinforcement Learning
A numerical study of the point-to-point navigation of a fish-like swimmer in a time-varying vortical flow with a hybrid method of deep reinforcement learning (DRL) and immersed boundary–lattice Boltzmann method (IB-LBM).
Stabilizing viscous extensional flows using reinforcement learning
The four-roll mill, wherein four identical cylinders undergo rotation of identical magnitude but alternate signs, was originally proposed by G. I. Taylor to create local extensional flows and study
Learning efficient navigation in vortical flow fields
A recently introduced Reinforcement Learning algorithm is applied to discover time-efficient navigation policies to steer a fixed-speed swimmer through unsteady two-dimensional flow fields and finds that the resulting swimmers successfully exploit the background flow to reach the target, but that this success depends on the sensed environmental cue.
Stable Schooling Formations Emerge from the Combined Effect of the Active Control and Passive Self-Organization
This work presents a numerical study of the collective motion of two freely-swimming swimmers by a hybrid method of the deep reinforcement learning method (DRL) and the immersed boundary-lattice
Optimizing low-Reynolds-number predation via optimal control and reinforcement learning
Abstract We seek the best stroke sequences of a finite-size swimming predator chasing a non-motile point or finite-size prey at low Reynolds number. We use optimal control to seek the globally
Mechanical rotation at low Reynolds number via reinforcement learning
The use of reinforcement learning is demonstrated to generate net mechanical rotation at low Reynolds numbers without requiring prior knowledge of locomotion.
Neural Circuit Architectural Priors for Embodied Control
This work translates C. elegans circuits for locomotion into an ANN model controlling a simulated Swimmer agent that achieves good initial performance and asymptotic performance comparable with MLPs, while dramatically improving data efficiency and requiring orders of magnitude fewer parameters.
Review of Metrics to Measure the Stability, Robustness and Resilience of Reinforcement Learning
Reinforcement learning (RL) has received significant interest in recent years, due primarily to the successes of deep reinforcement learning at solving many challenging tasks such as playing Chess,
Quantum support vector machines for aerodynamic classification
Aerodynamics plays an important role in aviation industry and aircraft design. Detecting and minimizing the phenomenon of flow separation from scattered pressure data on airfoil is critical for
Stabilising viscous extensional flows using Reinforcement Learning
It is shown that Reinforcement Learning can provide a control algorithm effective for all initial positions and that can be adapted to limit the magnitude of the flow extension near the position of the drop.


Learning to school in the presence of hydrodynamic interactions
It is proposed to use swimmers equipped with adaptive decision-making that adjust their gaits through a reinforcement learning algorithm in response to nonlinearly varying hydrodynamic loads to identify schooling patterns that minimize the individual and collective swimming effort.
Gait and speed selection in slender inertial swimmers
A minimal theoretical framework is presented that synthesizes the roles of passive body elasticity, hydrodynamics, muscular activation, and proprioceptive sensory feedback in inertial swimmers to provide a simple mechanistic view of swimming consistent with natural observations and suggest ways to engineer artificial swimmers for optimal performance.
Finding efficient swimming strategies in a three-dimensional chaotic flow by reinforcement learning
A reinforcement learning algorithm is applied to show how smart particles can learn approximately optimal strategies to navigate in complex flows with chaotic advection along streamlines and it is shown that it is sufficient to endow the swimmers with a very restricted set of actions to have enough freedom to find efficient strategies to move upward and escape local fluid traps.
On the best design for undulatory swimming
  • C. Eloy
  • Physics
    Journal of Fluid Mechanics
  • 2013
Abstract Most aquatic vertebrates swim by passing a bending wave down their bodies, a swimming mode known as undulatory propulsion. Except for very elongated swimmers like eels and lampreys, these
Vortex phase matching as a strategy for schooling in robots and in fish
Bio-mimetic fish-like robots are developed which allow us to measure directly the energy consumption associated with swimming together in pairs and find that followers, in any relative position to a near-neighbour, could obtain hydrodynamic benefits if they exhibit a tailbeat phase difference that varies linearly with front-back distance.
Fishlike rheotaxis
A theoretical model consisting of a fishlike body equipped with lateral pressure sensors in oncoming uniform flows is proposed and it is found that, as in fish rheotaxis, the fish like body responds by aligning into the oncoming flow.
Efficient collective swimming by harnessing vortices through deep reinforcement learning
This study confirms that fish may harvest energy deposited in vortices and support the conjecture that swimming in formation is energetically advantageous and demonstrates that deep RL can produce navigation algorithms for complex unsteady and vortical flow fields, with promising implications for energy savings in autonomous robotic swarms.
Self-learning how to swim at low Reynolds number
This work demonstrates theoretically that this first self-learning swimmer can recover a previously known propulsion strategy without prior knowledge in low Re locomotion, identify more effective locomotory gaits when the number of spheres increases, and adapt its locomotORY gaits in different media.
Inferring the structure and dynamics of interactions in schooling fish
Comparing data from two-fish and three-fish shoals challenges the standard assumption that individual motion results from averaging responses to each neighbor considered separately and finds no evidence for explicit matching of body orientation.
Simulations of optimized anguilliform swimming
The results of the present simulations support the hypothesis that anguilliform swimmers modify their kinematics according to different objectives and provide a quantitative analysis of the swimming motion and the forces experienced by the body.