Corpus ID: 2159236

An Application of Inverse Reinforcement Learning to Medical Records of Diabetes Treatment

  title={An Application of Inverse Reinforcement Learning to Medical Records of Diabetes Treatment},
  author={Hideki Asoh and Masanori Shiro and Shotaro Akaho and Toshihiro Kamishima},
It is an important issue to utilize large amount of medical records which are being accumulated on medical information systems to improve the quality of medical treatment. The process of medical treatment can be considered as a sequential interaction process between doctors and patients. From this viewpoint, we have been modeling medical records using Markov decision processes (MDPs). Using our model, we can simulate the future of each patient and evaluate each treatment. In order to do so, the… Expand
Optimal medication dosing from suboptimal clinical examples: A deep reinforcement learning approach
  • S. Nemati, M. Ghassemi, G. Clifford
  • Computer Science, Medicine
  • 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)
  • 2016
This work employed retrospective data from the publicly available MIMIC II intensive care unit database, and developed a deep reinforcement learning algorithm that learns an optimal heparin dosing policy from sample dosing trails and their associated outcomes in large electronic medical records. Expand
Deep Reinforcement Learning for Clinical Decision Support: A Brief Survey
This is the first survey that summarizes reinforcement learning algorithms with Deep Neural Networks (DNN) on clinical decision support and compares and contrast the advantages and limitations of various DRL algorithms and presents a preliminary guide on how to choose the appropriate DRL algorithm for particular clinical applications. Expand
Optimizing Sequential Medical Treatments with Auto-Encoding Heuristic Search in POMDPs
This work model patient-clinician interactions as partially observable Markov decision processes (POMDPs) and optimize sequential treatment based on belief states inferred from history sequence to build a variational generative model and boost state representation with a recurrent neural network (RNN). Expand
Reinforcement Learning in Healthcare: A Survey
This survey provides an extensive overview of RL applications in a variety of healthcare domains, ranging from dynamic treatment regimes in chronic diseases and critical care, automated medical diagnosis, and many other control or scheduling problems that have infiltrated every aspect of the healthcare system. Expand
Optimizing Medical Treatment for Sepsis in Intensive Care: from Reinforcement Learning to Pre-Trial Evaluation
This work learns off-policy continuous dosing policy of pharmaceuticals for sepsis treatment using historical intensive care data under partially observable MDPs (POMDPs), and evaluates the model recommendations using not only conventional policy evaluations but a novel framework that incorporates human experts. Expand
Learning models for writing better doctor prescriptions
A data-driven approach for learning and improving the prescription policy physicians use to treat Type 2 diabetes that combines regression, classification and strategy optimization and offers a better therapeutic effect than the state-of-art deterministic algorithms. Expand
Representation and Reinforcement Learning for Personalized Glycemic Control in Septic Patients
The result demonstrates that reinforcement learning with appropriate patient state encoding can potentially provide optimal glycemic trajectories and allow clinicians to design a personalized strategy for glycemic control in septic patients. Expand
The Actor Search Tree Critic (ASTC) for Off-Policy POMDP Learning in Medical Decision Making
This work captures partially observable Markov decision process, in which an agent optimises its actions in a belief represented as a distribution of patient states inferred from individual history trajectories, and proposes a Gaussian mixture model fitted for the observed data. Expand
Task-Guided Inverse Reinforcement Learning Under Partial Information
An algorithm for IRL in partially observable Markov decision processes (POMDPs), where an agent cannot directly observe the current state of the POMDP, is developed and it is demonstrated that, even with severely limited data, the algorithm learns reward functions and policies that satisfy the task and induce a similar behavior to the expert by leveraging the side information and incorporating memory into the policy. Expand
State Distribution-Aware Sampling for Deep Q-Learning
This work proposes a novel state distribution-aware sampling method to balance the replay times for transitions with imbalanced distribution, which takes into account both the occurrence frequencies of transitions and the uncertainty of state-action values. Expand


Modeling Medical Records of Diabetes using Markov Decision Processes
Analyzing long-term medical records of patients suffering from chronic diseases is beginning to be recognized as an important issue in medical data analysis. Long term medical treatments can beExpand
Apprenticeship learning via inverse reinforcement learning
This work thinks of the expert as trying to maximize a reward function that is expressible as a linear combination of known features, and gives an algorithm for learning the task demonstrated by the expert, based on using "inverse reinforcement learning" to try to recover the unknown reward function. Expand
Bayesian Inverse Reinforcement Learning
This paper shows how to combine prior knowledge and evidence from the expert's actions to derive a probability distribution over the space of reward functions and presents efficient algorithms that find solutions for the reward learning and apprenticeship learning tasks that generalize well over these distributions. Expand
Preference elicitation and inverse reinforcement learning
It is shown that preferences can be determined accurately, even if the observed agent's policy is sub-optimal with respect to its own preferences, and the relation of the resulting approach to other statistical methods for inverse reinforcement learning is examined. Expand
Bayesian Multitask Inverse Reinforcement Learning
The main contribution is to formalise the problem of inverse reinforcement learning as statistical preference elicitation, via a number of structured priors, whose form captures the authors' biases about the relatedness of different tasks or expert policies. Expand
Algorithms for Inverse Reinforcement Learning
Pharmacokinetics of ivermectin after IV administration were best described by a 2-compartment open model; values for main compartmental variables included volume of distribution at a steady state, area under the plasma concentration-time curve, and area underThe AUC curve. Expand
Artificial Intelligence: A Modern Approach
The long-anticipated revision of this #1 selling book offers the most comprehensive, state of the art introduction to the theory and practice of artificial intelligence for modern applications.Expand
Markov Decision Processes
The theory of Markov Decision Processes is the theory of controlled Markov chains. Its origins can be traced back to R. Bellman and L. Shapley in the 1950’s. During the decades of the last centuryExpand
A Markovian Decision Process
Abstract : The purpose of this paper is to discuss the asymptotic behavior of the sequence (f sub n(i)) generated by a nonlinear recurrence relation. This problem arises in connection with anExpand
Kohro: Modeling medical records of diabetes using Markov decision processes
  • Proceedings of the ICML2013 Workshop on Roll of Machine Learning for Transforming Healthcare
  • 2013