Markov State Models: Theory and Algorithms

Many complex systems exhibit metastable dynamical behavior. Metastable system have regions in state space that are attractive for the dynamics in the sense that typical trajectories remain within such regions for long periods of time before making the transition towards another metastable sets.

In recent years, analysis techniques for such processes have been developed. So-called Markov State Models (MSM) have been particularly successful recently since it could be demonstrated that MSMs can be constructed even for very high dimensional systems and have been especially useful for modeling then interesting slow dynamics of biomolecules and materials. MSMs rely on the (fuzzy) subdivision of state space of the system into discrete substates.  A MSM is given by the transition probabilities between these substates. They are estimated based on simulation data of the original process. The term Markov model indicates that the kinetics are modeled by a memoryless jump process between states.

The approximation error of MSMs contains two different contributions: (A) the discretization error related to the choice of the (fuzzy) subdivision underlying the MSM, and (B) the sampling error caused by the finiteness of the trajectory information from which the transition probabilities are computed.

Our aim is to develop reliable estimates of these two components of the approximation error independent of any timescale separation or other asymptotic assumptions, and to construct efficient algorithms for the construction of MSMs for high-dimensional complex systems.


The project is funded through MATHEON project A19.

Selected Publication

  • Sarich, M. and Noe, F. and Schuette, Ch. (2010) On the Approximation Quality of Markov State Models. Multiscale Model. Simul., 8 (4). pp. 1154-1177.


  • Metzner, Ph. and Noe, F. and Schuette, Ch. (2009) Estimating the Sampling Error: Distribution of Transition Matrices and Functions of Transition Matrices for Given Trajectory Data. Phys. Rev. E, 80 (2). 021106.


  • Djurdjevac, N. and Sarich, M. and Schuette, Ch. (2010) On Markov State Models for Metastable Processes. In: Proceedings of ICM 2010. Section Invited Talks.


Christof Schütte

Project researchers

Marco Sarich, Natasa Djurdjevac, Frank Noe, Christof Schuette, Jan-Hendrik Prinz, Steffi Winkelmann



P. Deuflhard and M. Weber (ZIB), E. Vanden Eijnden (Courant, NYU), J. Chodera (Berkeley)