0

Let's consider a markov chain that has no transient states and is irreducible.

I am struggling if there is really any interpretation from it.

For example, suppose it is possible to return to the state in {6, 8, 10, 12, ...} time steps; the period would be 2, but 2 does not appear in this list. So what does this '2' actually mean?

Also, suppose it is possible to return to the state in {$2m+3n$ where $m$, $n$ are positive integers } time steps; the period would be 1 but I don't see if there is any signifiant difference between the 2 cases even though one is periodic while the other one is aperiodic.

Many thanks.

Anson NG
  • 965

1 Answers1

0

If the period of an irreducible, positive recurrent Markov chain is 1, then, in the long run, it is possible for the chain to be in any state at any time, regardless of the initial distribution. However, if the period is $2$, this is no longer true. If, for example, the initial distribution assigns probability 1 to one state, then some states will only be visited after an even number of time steps, while the other states will only be visited after an odd number of time steps.

Consider the simplest chain with period equal to 2: two states forming a ring and connected by transitions with probability 1. If the initial distribution assigns probability 1/2 to each state, then a stationary distribution is reached. (In fact, the initial distribution is the stationary distribution.)

However, if an irreducible, positive recurrent Markov chain has period 1, then, for sufficiently large $n$, all entries of the $n$-th power of the transition matrix are strictly positive, and for any initial distribution, convergence is guaranteed to the stationary distribution.

Fabio Somenzi
  • 7,676