Definition and Examples of DTMCs

The objective of this chapter is to describe a special type of stochastic process called discrete time Markov chain (DTMC). We first provide definitions for DTMCs using simple language as well as mathematics. Then we describe a systematic technique to model a system as a DTMC. Finally we provide examples to both illustrate the modeling technique as well as… (More)