# Reinforcement Learning in Economics and Finance

@article{Charpentier2021ReinforcementLI, title={Reinforcement Learning in Economics and Finance}, author={Arthur Charpentier and Romuald Elie and Carl Remlinger}, journal={ArXiv}, year={2021}, volume={abs/2003.10014} }

Reinforcement learning algorithms describe how an agent can learn an optimal action policy in a sequential decision process, through repeated experience. In a given environment, the agent policy provides him some running and terminal rewards. As in online learning, the agent learns sequentially. As in multi-armed bandit problems, when an agent picks an action, he can not infer ex-post the rewards induced by other action choices. In reinforcement learning, his actions have consequences: they…

## 40 Citations

### Learning from Zero: How to Make Consumption-Saving Decisions in a Stochastic Environment with an AI Algorithm

- EconomicsSSRN Electronic Journal
- 2021

This exercise offers an innovative learning mechanism to model economic agent’s decision-making process using a deep reinforcement learning algorithm. In particular, this AI agent is born in an…

### Learning to make consumption-saving decisions in a changing environment: an AI approach

- Economics
- 2021

This exercise offers an innovative learning mechanism to model economic agent’s decision-making process using a deep reinforcement learning algorithm. In particular, this AI agent has limited or no…

### Reinforcement Learning Approaches to Optimal Market Making

- Computer ScienceMathematics
- 2021

The analysis indicated that reinforcement learning techniques provide superior performance in terms of the risk-adjusted return over more standard market making strategies, typically derived from analytical models.

### Winning at Any Cost - Infringing the Cartel Prohibition With Reinforcement Learning

- Computer SciencePAAMS
- 2021

This work builds a scenario that is based on a modified version of a prisoner’s dilemma where three agents play the game of rock paper scissors and indicates that the action selection can be dissected into specific stages, establishing the possibility to develop collusion prevention systems that are able to recognize situations which might lead to a collusion between competitors.

### Pseudo-Model-Free Hedging for Variable Annuities via Deep Reinforcement Learning

- Computer Science
- 2021

This paper proposes a two-phase deep reinforcement learning approach, for hedging variable annuity contracts with both GMMB and GMDB riders, which can address model miscalibration in Black-Scholes…

### Reinforcement Learning for Mean Field Games, with Applications to Economics

- EconomicsArXiv
- 2021

A two timescale approach with RL for MFG and MFC, which relies on a unified Q-learning algorithm to simultaneously update an action-value function and a distribution but with different rates, in a model-free fashion.

### Multi-Agent Inverse Reinforcement Learning: Suboptimal Demonstrations and Alternative Solution Concepts

- BiologyArXiv
- 2021

It was found that the primary methods for handling noise, biases and heuristics in MIRL were extensions of Maximum Entropy IRL to multi-agent settings and traditional Nash equilibrium solution concepts were ill-suited to model human behavior.

### Recent Advances in Deep Reinforcement Learning Applications for Solving Partially Observable Markov Decision Processes (POMDP) Problems Part 2 - Applications in Transportation, Industries, Communications and Networking and More Topics

- Computer ScienceMach. Learn. Knowl. Extr.
- 2021

A survey on recent advances in Deep Reinforcement Learning (DRL) for solving partially observable Markov decision processes (POMDP) problems and applications of DRL for solving POMDP problems in games, robotics, and natural language processing.

### A Survey on Large-Population Systems and Scalable Multi-Agent Reinforcement Learning

- Computer ScienceArXiv
- 2022

This survey sheds light on current approaches to tractably understanding and analyzing large-population systems, both through multi-agent reinforcement learning and through adjacent areas of research such as mean-ﬁeld games, collective intelligence, or complex network theory.

### Optimal Monetary Policy Using Reinforcement Learning

- EconomicsSSRN Electronic Journal
- 2021

This paper introduces a reinforcement learning based approach to compute optimal interest rate reaction functions in terms of fulfilling inflation and output gap targets. The method is generally…

## References

SHOWING 1-10 OF 168 REFERENCES

### Applying reinforcement learning to economic problems

- Computer Science
- 2014

This paper introduces methods based around fitted Q iteration, a batch version of Q-learning, and considers how reinforcement learning can be applied to complex multiagent problems: stochastic games, where each agent faces a MDP with transition and payoff functions that are dependent on the actions of the other players.

### Reinforcement Learning: An Introduction

- Computer ScienceIEEE Transactions on Neural Networks
- 2005

This book provides a clear and simple account of the key ideas and algorithms of reinforcement learning, which ranges from the history of the field's intellectual foundations to the most recent developments and applications.

### Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms

- Computer ScienceArXiv
- 2019

This chapter reviews the theoretical results of MARL algorithms mainly within two representative frameworks, Markov/stochastic games and extensive-form games, in accordance with the types of tasks they address, i.e., fully cooperative, fully competitive, and a mix of the two.

### Apprenticeship learning via inverse reinforcement learning

- Computer ScienceICML
- 2004

This work thinks of the expert as trying to maximize a reward function that is expressible as a linear combination of known features, and gives an algorithm for learning the task demonstrated by the expert, based on using "inverse reinforcement learning" to try to recover the unknown reward function.

### Learning to trade via direct reinforcement

- Computer Science, EconomicsIEEE Trans. Neural Networks
- 2001

It is demonstrated how direct reinforcement can be used to optimize risk-adjusted investment returns (including the differential Sharpe ratio), while accounting for the effects of transaction costs.

### Reinforcement Learning for Market Making in a Multi-agent Dealer Market

- EconomicsArXiv
- 2019

It is shown that the reinforcement learning agent is able to learn about its competitor's pricing policy and it also learns to manage inventory by smartly selecting asymmetric prices on the buy and sell sides, and maintaining a positive or negative inventory depending on whether the market price drift is positive (or negative).

### On the Convergence of Model Free Learning in Mean Field Games

- Computer ScienceAAAI
- 2020

This paper analyzes in full generality the convergence of a fictitious iterative scheme using any single agent learning algorithm at each step of the Mean Field MAS, and shows for the first time convergence of model free learning algorithms towards non-stationary MFG equilibria.

### Preference elicitation and inverse reinforcement learning

- EconomicsECML/PKDD
- 2011

It is shown that preferences can be determined accurately, even if the observed agent's policy is sub-optimal with respect to its own preferences, and the relation of the resulting approach to other statistical methods for inverse reinforcement learning is examined.