Efficient maximum likelihood parameterization of continuous-time Markov processes.

@article{McGibbon2015EfficientML,
  title={Efficient maximum likelihood parameterization of continuous-time Markov processes.},
  author={Robert T. McGibbon and Vijay S. Pande},
  journal={The Journal of chemical physics},
  year={2015},
  volume={143 3},
  pages={
          034109
        }
}
Continuous-time Markov processes over finite state-spaces are widely used to model dynamical processes in many fields of natural and social science. Here, we introduce a maximum likelihood estimator for constructing such models from data observed at a finite time interval. This estimator is dramatically more efficient than prior approaches, enables the calculation of deterministic confidence intervals in all model parameters, and can easily enforce important physical constraints on the models… 
Inferring Transition Rates of Networks from Populations in Continuous-Time Markov Processes.
TLDR
The present work leads to a particularly important analytical result: namely, that when the network is constrained only by a mean jump rate, the rate matrix is given by a square-root dependence of the rate, kab ∝ (πb/πa)(1/2), on πa and πb, the stationary-state populations at nodes a and b.
Learning Continuous-Time Hidden Markov Models for Event Data
TLDR
Recent advances in the development of efficient EM-based learning methods for CT-HMM models are described and the use of models with more than 100 states to analyze disease progression using glaucoma and Alzheimer’s Disease datasets is demonstrated.
The pathway elaboration method for mean first passage time estimation in large continuous-time Markov chains with applications to nucleic acid kinetics
TLDR
The pathway elaboration method is proposed, a time-efficient probabilistic truncationbased approach for detailed-balance CTMCs that can be used for estimating the MFPT for rare events in addition to rapidly evaluating perturbed parameters without expensive recomputations.
Efficient Learning and Decoding of the Continuous-Time Hidden Markov Model for Disease Progression Modeling
TLDR
This paper presents the first complete characterization of efficient EM-based learning methods for CT-HMM models, as well as the first solution to decoding the optimal state transition sequence and the corresponding state dwelling time.
Nearly reducible finite Markov chains: Theory and algorithms.
TLDR
All of the information required to determine the kinetically relevant transition mechanisms, and to identify the states that have a dominant effect on the global dynamics, can be computed reliably even for computationally challenging models.
Dimensional reduction of Markov state models from renormalization group theory.
TLDR
A lower dimensional stochastic model is constructed which provides the optimal coarse-grained Markovian representation of the system's relaxation kinetics and the computational cost of computing the low-dimensional model remains affordable on a desktop computer even for thousands of microstates.
Optimized parameter selection reveals trends in Markov state models for protein folding.
TLDR
This work presents a set of guidelines for constructing Markov state models of protein folding, and recommends the use of cross-validation and a kinetically motivated dimensionality reduction step for improved descriptions of folding dynamics.
Non-equilibrium Markov state modeling of periodically driven biomolecules.
TLDR
This work introduces a method to construct Markov state models for systems that are driven through periodically changing one (or several) external parameter, and illustrates the method for alanine dipeptide exposed to a time-dependent electric field.
Markov State Models: From an Art to a Science.
TLDR
An overview of the MSM field to date is presented, presented for a general audience as a timeline of key developments in the field, and the current frontiers of methods development are highlighted, as well as exciting applications in experimental design and drug discovery.
...
...

References

SHOWING 1-10 OF 111 REFERENCES
Statistical inference for discretely observed Markov jump processes
Summary.  Likelihood inference for discretely observed Markov jump processes with finite state space is investigated. The existence and uniqueness of the maximum likelihood estimator of the intensity
Efficient Bayesian estimation of Markov model transition matrices with given stationary distribution.
Direct simulation of biomolecular dynamics in thermal equilibrium is challenging due to the metastable nature of conformation dynamics and the computational cost of molecular dynamics. Biased or
On the Approximation Quality of Markov State Models
TLDR
A sharp error bound for the difference in propagation of probability densities between the MSM and the original process on long timescales is provided for a rather general class of Markov processes ranging from diffusions in energy landscapes to Markov jump processes on large discrete spaces.
Data-Based Inference of Generators for Markov Jump Processes Using Convex Optimization
TLDR
A variational approach to the estimation of generators for Markov jump processes from discretely sampled data is discussed and generalized and numerical aspects of the algorithm for estimation of processes with high-dimensional state spaces are discussed.
Counting labeled transitions in continuous-time Markov models of evolution
TLDR
It is demonstrated that it is possible to obtain closed-form analytic solutions for the probability mass and probability generating functions of this evolutionary counting process using an eigen decomposition of the infinitesimal generator, provided the latter is a diagonalizable matrix.
Markov models of molecular kinetics: generation and validation.
TLDR
An upper bound for the approximation error made by modeling molecular dynamics with a Markov chain is described and it is shown that this error can be made arbitrarily small with surprisingly little effort.
Statistical model selection for Markov models of biomolecular dynamics.
TLDR
Application of techniques that consider both systematic bias and statistical error on two 100 μs molecular dynamics trajectories of the Fip35 WW domain shows agreement with existing techniques based on self-consistency of the model's relaxation time scales with more suitable results in regimes in which those time scale-based techniques encourage overfitting.
Continuous Time Markov Chain Models for Chemical Reaction Networks
TLDR
This chapter develops much of the mathematical machinery needed to describe the stochastic models of reaction networks and shows how to derive the deterministic law of mass action from the Markov chain model.
...
...