• Corpus ID: 53716309

Model-Based Reinforcement Learning for Sepsis Treatment

@article{Raghu2018ModelBasedRL,
  title={Model-Based Reinforcement Learning for Sepsis Treatment},
  author={Aniruddh Raghu and Matthieu Komorowski and Sumeetpal S. Singh},
  journal={ArXiv},
  year={2018},
  volume={abs/1811.09602}
}
Sepsis is a dangerous condition that is a leading cause of patient mortality. Treating sepsis is highly challenging, because individual patients respond very differently to medical interventions and there is no universally agreed-upon treatment for sepsis. In this work, we explore the use of continuous state-space model-based reinforcement learning (RL) to discover high-quality treatment policies for sepsis patients. Our quantitative evaluation reveals that by blending the treatment strategy… 

Figures and Tables from this paper

Reinforcement Learning for Sepsis Treatment: Baselines and Analysis

TLDR
The results illustrate that simple, tabular Q-learning and Deep Q-Learning both lead to the most effective medical treatment strategies, and that temporal encoding in the state representation aids in discovering improved policies.

An Empirical Study of Representation Learning for Reinforcement Learning in Healthcare

TLDR
It is found that sequentially formed state representations facilitate effective policy learning in batch settings, validating a more thoughtful approach to representation learning that remains faithful to the sequential and partial nature of healthcare data.

Challenges for Reinforcement Learning in Healthcare

TLDR
A reinforcement learning agent could be trained to provide treatment recommendations for physicians, acting as a decision support tool, but a number of difficulties arise when using RL beyond benchmark environments, such as specifying the reward function, choosing an appropriate state representation and evaluating the learned policy.

Understanding the complexity of sepsis mortality prediction via rule discovery and analysis: a pilot study

TLDR
Glasgow Coma Scale, serum potassium, and serum bilirubin are found to be the most important risk factors for predicting patient death in sepsis patients.

Applications of Reinforcement learning for Medical Decision Making

TLDR
Various RL applications that would be effective in improving the existing healthcare sector at same time being efficient in handling complex medical decision-making tasks are illustrated.

An Empirical Study of Representation Learning for Reinforcement Learning in Healthcare

TLDR
It is found that sequentially formed state representations facilitate effective policy learning in batch settings, validating a more thoughtful approach to representation learning that ∗ Equal Cont contribution † Equal Contribution remains faithful to the sequential and partial nature of healthcare data.

MGP-AttTCN: An interpretable machine learning model for the prediction of sepsis

TLDR
This work proposes MGP-AttTCN: a joint multitask Gaussian Process and attention-based deep learning model to early predict the occurrence of sepsis in an interpretable manner and shows that the model outperforms the current state-of-the-art and presents evidence that different labelling heuristics lead to discrepancies in task difficulty.

Estimating Treatment Effects for Time-to-Treatment Antibiotic Stewardship in Sepsis

TLDR
T4 estimates individual treatment effects (ITEs) by recurrently encoding temporal and static variables as potential confounders, and then decoding the outcomes under different treatment sequences and achieves interpretability through a global-level attention mechanism and a variable-level importance examination.

References

SHOWING 1-10 OF 25 REFERENCES

Continuous State-Space Models for Optimal Sepsis Treatment: a Deep Reinforcement Learning Approach

TLDR
A new approach to deduce optimal treatment policies for septic patients by using continuous state-space models and deep reinforcement learning is proposed, which is able to learn clinically interpretable treatment policies, similar in important aspects to the treatment policies of physicians.

Evaluating Reinforcement Learning Algorithms in Observational Health Settings

TLDR
The goal is to expose some of the subtleties associated with evaluating RL algorithms in healthcare to provide a conceptual starting point for clinical and computational researchers to ask the right questions when designing and evaluating algorithms for new ways of treating patients.

Surviving Sepsis Campaign: International Guidelines for Management of Sepsis and Septic Shock 2016

TLDR
Although a significant number of aspects of care have relatively weak support, evidence-based recommendations regarding the acute management of sepsis and septic shock are the foundation of improved outcomes for these critically ill patients with high mortality.

Surviving Sepsis Campaign: International Guidelines for Management of Severe Sepsis and Septic Shock 2012

TLDR
An update to the “Surviving Sepsis Campaign Guidelines for Management of Severe Sepsis and Septic Shock,” last published in 2008 is provided.

The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3).

TLDR
The task force concluded the term severe sepsis was redundant and updated definitions and clinical criteria should replace previous definitions, offer greater consistency for epidemiologic studies and clinical trials, and facilitate earlier recognition and more timely management of patients with sepsi or at risk of developing sepsic shock.

Interaction Between Fluids and Vasoactive Agents on Mortality in Septic Shock: A Multicenter, Observational Study*

TLDR
The focus during the first hour of resuscitation for septic shock should beaggressive fluid administration, only thereafter starting vasoactive agents, while continuing aggressive fluid administration.

Learning and Policy Search in Stochastic Dynamical Systems with Bayesian Neural Networks

TLDR
An algorithm for model-based reinforcement learning that combines Bayesian neural networks (BNNs) with random roll-outs and stochastic optimization for policy learning and achieves promising results in a real-world scenario for controlling a gas turbine.

Sepsis in European intensive care units: Results of the SOAP study*

TLDR
This large pan-European study documents the high frequency of sepsis in critically ill patients and shows a close relationship between the proportion of patients with sepsi and the intensive care unit mortality in the various countries.

MIMIC-III, a freely accessible critical care database

MIMIC-III (‘Medical Information Mart for Intensive Care’) is a large, single-center database comprising information relating to patients admitted to critical care units at a large tertiary care

Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuning

TLDR
It is demonstrated that neural network dynamics models can in fact be combined with model predictive control (MPC) to achieve excellent sample complexity in a model-based reinforcement learning algorithm, producing stable and plausible gaits that accomplish various complex locomotion tasks.