Reinforcement Learning in Economics and Finance

  title={Reinforcement Learning in Economics and Finance},
  author={Arthur Charpentier and Romuald Elie and Carl Remlinger},
Reinforcement learning algorithms describe how an agent can learn an optimal action policy in a sequential decision process, through repeated experience. In a given environment, the agent policy provides him some running and terminal rewards. As in online learning, the agent learns sequentially. As in multi-armed bandit problems, when an agent picks an action, he can not infer ex-post the rewards induced by other action choices. In reinforcement learning, his actions have consequences: they… 

Learning from Zero: How to Make Consumption-Saving Decisions in a Stochastic Environment with an AI Algorithm

  • Rui Shi
  • Economics
    SSRN Electronic Journal
  • 2021
This exercise offers an innovative learning mechanism to model economic agent’s decision-making process using a deep reinforcement learning algorithm. In particular, this AI agent is born in an

Learning to make consumption-saving decisions in a changing environment: an AI approach

This exercise offers an innovative learning mechanism to model economic agent’s decision-making process using a deep reinforcement learning algorithm. In particular, this AI agent has limited or no

Reinforcement Learning Approaches to Optimal Market Making

The analysis indicated that reinforcement learning techniques provide superior performance in terms of the risk-adjusted return over more standard market making strategies, typically derived from analytical models.

Winning at Any Cost - Infringing the Cartel Prohibition With Reinforcement Learning

This work builds a scenario that is based on a modified version of a prisoner’s dilemma where three agents play the game of rock paper scissors and indicates that the action selection can be dissected into specific stages, establishing the possibility to develop collusion prevention systems that are able to recognize situations which might lead to a collusion between competitors.

Pseudo-Model-Free Hedging for Variable Annuities via Deep Reinforcement Learning

This paper proposes a two-phase deep reinforcement learning approach, for hedging variable annuity contracts with both GMMB and GMDB riders, which can address model miscalibration in Black-Scholes

Reinforcement Learning for Mean Field Games, with Applications to Economics

A two timescale approach with RL for MFG and MFC, which relies on a unified Q-learning algorithm to simultaneously update an action-value function and a distribution but with different rates, in a model-free fashion.

Multi-Agent Inverse Reinforcement Learning: Suboptimal Demonstrations and Alternative Solution Concepts

It was found that the primary methods for handling noise, biases and heuristics in MIRL were extensions of Maximum Entropy IRL to multi-agent settings and traditional Nash equilibrium solution concepts were ill-suited to model human behavior.

Recent Advances in Deep Reinforcement Learning Applications for Solving Partially Observable Markov Decision Processes (POMDP) Problems Part 2 - Applications in Transportation, Industries, Communications and Networking and More Topics

A survey on recent advances in Deep Reinforcement Learning (DRL) for solving partially observable Markov decision processes (POMDP) problems and applications of DRL for solving POMDP problems in games, robotics, and natural language processing.

A Survey on Large-Population Systems and Scalable Multi-Agent Reinforcement Learning

This survey sheds light on current approaches to tractably understanding and analyzing large-population systems, both through multi-agent reinforcement learning and through adjacent areas of research such as mean-field games, collective intelligence, or complex network theory.

Optimal Monetary Policy Using Reinforcement Learning

This paper introduces a reinforcement learning based approach to compute optimal interest rate reaction functions in terms of fulfilling inflation and output gap targets. The method is generally



Applying reinforcement learning to economic problems

This paper introduces methods based around fitted Q iteration, a batch version of Q-learning, and considers how reinforcement learning can be applied to complex multiagent problems: stochastic games, where each agent faces a MDP with transition and payoff functions that are dependent on the actions of the other players.

On the Computational Economics of Reinforcement Learning

Reinforcement Learning: An Introduction

This book provides a clear and simple account of the key ideas and algorithms of reinforcement learning, which ranges from the history of the field's intellectual foundations to the most recent developments and applications.

Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms

This chapter reviews the theoretical results of MARL algorithms mainly within two representative frameworks, Markov/stochastic games and extensive-form games, in accordance with the types of tasks they address, i.e., fully cooperative, fully competitive, and a mix of the two.

Markov Games as a Framework for Multi-Agent Reinforcement Learning

Apprenticeship learning via inverse reinforcement learning

This work thinks of the expert as trying to maximize a reward function that is expressible as a linear combination of known features, and gives an algorithm for learning the task demonstrated by the expert, based on using "inverse reinforcement learning" to try to recover the unknown reward function.

Learning to trade via direct reinforcement

It is demonstrated how direct reinforcement can be used to optimize risk-adjusted investment returns (including the differential Sharpe ratio), while accounting for the effects of transaction costs.

Reinforcement Learning for Market Making in a Multi-agent Dealer Market

It is shown that the reinforcement learning agent is able to learn about its competitor's pricing policy and it also learns to manage inventory by smartly selecting asymmetric prices on the buy and sell sides, and maintaining a positive or negative inventory depending on whether the market price drift is positive (or negative).

On the Convergence of Model Free Learning in Mean Field Games

This paper analyzes in full generality the convergence of a fictitious iterative scheme using any single agent learning algorithm at each step of the Mean Field MAS, and shows for the first time convergence of model free learning algorithms towards non-stationary MFG equilibria.

Preference elicitation and inverse reinforcement learning

It is shown that preferences can be determined accurately, even if the observed agent's policy is sub-optimal with respect to its own preferences, and the relation of the resulting approach to other statistical methods for inverse reinforcement learning is examined.