Corpus ID: 174803358

Risk-Sensitive Compact Decision Trees for Autonomous Execution in Presence of Simulated Market Response

  title={Risk-Sensitive Compact Decision Trees for Autonomous Execution in Presence of Simulated Market Response},
  author={Svitlana Vyetrenko and Shaojie Xu},
We demonstrate an application of risk-sensitive reinforcement learning to optimizing execution in limit order book markets. We represent taking order execution decisions based on limit order book knowledge by a Markov Decision Process; and train a trading agent in a market simulator, which emulates multi-agent interaction by synthesizing market response to our agent's execution decisions from historical data. Due to market impact, executing high volume orders can incur significant cost. We… Expand
Learning who is in the market from time series: market participant discovery through adversarial calibration of multi-agent simulators
A novel two-step method is proposed to train a discriminator that is able to distinguish between “real” and “fake” price and volume time series as a part of GAN with self-attention, and then utilize it within an optimization framework to tune parameters of a simulator model with known agent archetypes to represent a market scenario. Expand
Reinforcement Learning in Economics and Finance
A state-of-the-art of reinforcement learning techniques are proposed, and applications in economics, game theory, operation research and finance are presented. Expand
Get real: realism metrics for robust limit order book market simulations
This paper surveyed the literature to collect a set of reference metrics and applied them to real market data and simulation output and provides a comprehensive catalog of these metrics including mathematical formulations where appropriate. Expand
Real-World Reinforcement Learning: Observations from Two Successful Cases
Reinforcement Learning (RL) is a machine learning technique that enables artificial agents to learn optimal strategies for sequential decision-making problems. RL has achieved superhuman performanceExpand
An overall view of key problems in algorithmic trading and recent progress
The key problems of algorithmic trading are presented, describing the concepts of optimal execution, optimal placement, and price impact and the most recent advances through the use of Machine Learning are discussed. Expand
The Microstructure of Cointegrated Assets
An execution algorithm is designed based on this micro price of two highly cointegrated assets, using Level-1 data collected on Interactive Brokers, and it is shown that it can save half of the bid-ask spread cost. Expand
Multi-agent reinforcement learning in a realistic limit order book market simulation
A model-free approach by training Reinforcement Learning (RL) agents in a realistic market simulation environment with multiple agents for optimal execution in an RL setting in which an intelligent agent can make order execution and placement decisions based on market microstructure trading signals in HFT. Expand
Robust Market Making via Adversarial Reinforcement Learning
It is shown that adversarial reinforcement learning (ARL) can be used to produce market marking agents that are robust to adversarial and adaptively-chosen market conditions and empirically demonstrate that the ARL method consistently converges. Expand


Market Making via Reinforcement Learning
A high-fidelity simulation of limit order book markets is developed, and a market making agent using temporal-difference reinforcement learning is designed using a linear combination of tile codings as a value function approximator and a custom reward function that controls inventory risk. Expand
Risk-Sensitive Reinforcement Learning
A risk-sensitive Q-learning algorithm is derived, which is necessary for modeling human behavior when transition probabilities are unknown, and applied to quantify human behavior in a sequential investment task and is found to provide a significantly better fit to the behavioral data and leads to an interpretation of the subject's responses that is indeed consistent with prospect theory. Expand
Risk-Sensitive Reinforcement Learning
This risk-sensitive reinforcement learning algorithm is based on a very different philosophy and reflects important properties of the classical exponential utility framework, but avoids its serious drawbacks for learning. Expand
Deep Hedging
We present a framework for hedging a portfolio of derivatives in the presence of market frictions such as transaction costs, market impact, liquidity constraints or risk limits using modern deepExpand
Machine Learning for Trading
In multi-period trading with realistic market impact, determining the dynamic trading strategy that optimizes expected utility of final wealth is a hard problem. In this paper we show that, with anExpand
Learning to trade via direct reinforcement
It is demonstrated how direct reinforcement can be used to optimize risk-adjusted investment returns (including the differential Sharpe ratio), while accounting for the effects of transaction costs. Expand
Optimal Execution with Limit and Market Orders
An optimal execution policy for an investor seeking to execute a large order using limit and market orders is developed and how the execution policies perform when targeting the volume schedule of the Almgren-Chriss execution strategy is shown. Expand
Enhancing Q-Learning for Optimal Asset Allocation
The proposed methods allows the construction of a multi-period portfolio management system which takes into account transaction costs, the risk preferences of the investor, and several constraints on the allocation. Expand
Reinforcement learning for optimized trade execution
We present the first large-scale empirical application of reinforcement learning to the important problem of optimized trade execution in modern financial markets. Our experiments are based on 1.5Expand
Optimal execution for portfolio transactions
In my thesis I explore the problem of optimizing trading strategies for complex portfolio transitions. Institutional investors run into this issue during periodic portfolio rebalancing or transitionExpand