# Efficient maximum likelihood parameterization of continuous-time Markov processes.

@article{McGibbon2015EfficientML, title={Efficient maximum likelihood parameterization of continuous-time Markov processes.}, author={Robert T. McGibbon and Vijay S. Pande}, journal={The Journal of chemical physics}, year={2015}, volume={143 3}, pages={ 034109 } }

Continuous-time Markov processes over finite state-spaces are widely used to model dynamical processes in many fields of natural and social science. Here, we introduce a maximum likelihood estimator for constructing such models from data observed at a finite time interval. This estimator is dramatically more efficient than prior approaches, enables the calculation of deterministic confidence intervals in all model parameters, and can easily enforce important physical constraints on the models…

## Figures from this paper

## 24 Citations

Maximum likelihood estimation of diffusions by continuous time Markov chain

- Mathematics, Computer ScienceComput. Stat. Data Anal.
- 2022

Inferring Transition Rates of Networks from Populations in Continuous-Time Markov Processes.

- Computer ScienceJournal of chemical theory and computation
- 2015

The present work leads to a particularly important analytical result: namely, that when the network is constrained only by a mean jump rate, the rate matrix is given by a square-root dependence of the rate, kab ∝ (πb/πa)(1/2), on πa and πb, the stationary-state populations at nodes a and b.

Learning Continuous-Time Hidden Markov Models for Event Data

- Computer ScienceMobile Health - Sensors, Analytic Methods, and Applications
- 2017

Recent advances in the development of efficient EM-based learning methods for CT-HMM models are described and the use of models with more than 100 states to analyze disease progression using glaucoma and Alzheimer’s Disease datasets is demonstrated.

The pathway elaboration method for mean first passage time estimation in large continuous-time Markov chains with applications to nucleic acid kinetics

- Computer Science
- 2021

The pathway elaboration method is proposed, a time-efficient probabilistic truncationbased approach for detailed-balance CTMCs that can be used for estimating the MFPT for rare events in addition to rapidly evaluating perturbed parameters without expensive recomputations.

Efficient Learning and Decoding of the Continuous-Time Hidden Markov Model for Disease Progression Modeling

- Computer ScienceArXiv
- 2021

This paper presents the first complete characterization of efficient EM-based learning methods for CT-HMM models, as well as the first solution to decoding the optimal state transition sequence and the corresponding state dwelling time.

Nearly reducible finite Markov chains: Theory and algorithms.

- Computer ScienceThe Journal of chemical physics
- 2021

All of the information required to determine the kinetically relevant transition mechanisms, and to identify the states that have a dominant effect on the global dynamics, can be computed reliably even for computationally challenging models.

Dimensional reduction of Markov state models from renormalization group theory.

- Computer ScienceThe Journal of chemical physics
- 2016

A lower dimensional stochastic model is constructed which provides the optimal coarse-grained Markovian representation of the system's relaxation kinetics and the computational cost of computing the low-dimensional model remains affordable on a desktop computer even for thousands of microstates.

Optimized parameter selection reveals trends in Markov state models for protein folding.

- Computer ScienceThe Journal of chemical physics
- 2016

This work presents a set of guidelines for constructing Markov state models of protein folding, and recommends the use of cross-validation and a kinetically motivated dimensionality reduction step for improved descriptions of folding dynamics.

Non-equilibrium Markov state modeling of periodically driven biomolecules.

- BiologyThe Journal of chemical physics
- 2019

This work introduces a method to construct Markov state models for systems that are driven through periodically changing one (or several) external parameter, and illustrates the method for alanine dipeptide exposed to a time-dependent electric field.

Markov State Models: From an Art to a Science.

- Computer ScienceJournal of the American Chemical Society
- 2018

An overview of the MSM field to date is presented, presented for a general audience as a timeline of key developments in the field, and the current frontiers of methods development are highlighted, as well as exciting applications in experimental design and drug discovery.

## References

SHOWING 1-10 OF 111 REFERENCES

Statistical inference for discretely observed Markov jump processes

- Mathematics
- 2005

Summary. Likelihood inference for discretely observed Markov jump processes with finite state space is investigated. The existence and uniqueness of the maximum likelihood estimator of the intensity…

Efficient Bayesian estimation of Markov model transition matrices with given stationary distribution.

- MathematicsThe Journal of chemical physics
- 2013

Direct simulation of biomolecular dynamics in thermal equilibrium is challenging due to the metastable nature of conformation dynamics and the computational cost of molecular dynamics. Biased or…

On the Approximation Quality of Markov State Models

- MathematicsMultiscale Model. Simul.
- 2010

A sharp error bound for the difference in propagation of probability densities between the MSM and the original process on long timescales is provided for a rather general class of Markov processes ranging from diffusions in energy landscapes to Markov jump processes on large discrete spaces.

Data-Based Inference of Generators for Markov Jump Processes Using Convex Optimization

- Computer Science, MathematicsMultiscale Model. Simul.
- 2009

A variational approach to the estimation of generators for Markov jump processes from discretely sampled data is discussed and generalized and numerical aspects of the algorithm for estimation of processes with high-dimensional state spaces are discussed.

Counting labeled transitions in continuous-time Markov models of evolution

- BiologyJournal of mathematical biology
- 2008

It is demonstrated that it is possible to obtain closed-form analytic solutions for the probability mass and probability generating functions of this evolutionary counting process using an eigen decomposition of the infinitesimal generator, provided the latter is a diagonalizable matrix.

Fitting timeseries by continuous-time Markov chains: A quadratic programming approach

- MathematicsJ. Comput. Phys.
- 2006

Markov models of molecular kinetics: generation and validation.

- PhysicsThe Journal of chemical physics
- 2011

An upper bound for the approximation error made by modeling molecular dynamics with a Markov chain is described and it is shown that this error can be made arbitrarily small with surprisingly little effort.

Statistical model selection for Markov models of biomolecular dynamics.

- Computer ScienceThe journal of physical chemistry. B
- 2014

Application of techniques that consider both systematic bias and statistical error on two 100 μs molecular dynamics trajectories of the Fip35 WW domain shows agreement with existing techniques based on self-consistency of the model's relaxation time scales with more suitable results in regimes in which those time scale-based techniques encourage overfitting.

Continuous Time Markov Chain Models for Chemical Reaction Networks

- Mathematics
- 2011

This chapter develops much of the mathematical machinery needed to describe the stochastic models of reaction networks and shows how to derive the deterministic law of mass action from the Markov chain model.