0

For an irreducible and aperiodic finite-state Markov chain, it has a limiting distribution (which turns out to be its unique stationary distribution), which is defined as the distribution over states at timestep $t$ as $t \rightarrow \infty$.

I've seen various places where people claim this distribution to also represent, e.g., "the fraction of time spent in each state in the long run". I find this confusing because the limiting distribution is defined as what happens at a single timestep that's "very distant" from now, and not in terms of long-term visitation.

How should I bridge the gap between these two concepts, the limiting distribution and the long-run visitation distribution? Intuitively, I think they are the same, but I couldn't convince myself mathematically.

Adam Wilson
  • 252
  • The relationship between the two is a nontrivial result called the ergodic theorem. It is in some sense a generalization of the law of large numbers. – Ian Oct 10 '22 at 16:59
  • Is this the theorem that you are referring to: https://mathworld.wolfram.com/BirkhoffsErgodicTheorem.html? – Adam Wilson Oct 10 '22 at 17:06
  • 1
    They're not exactly the same (Birkhoff's theorem is essentially this result but only for deterministic Markov processes). Still, they are related. You want the ergodic theorem for Markov chains. – Ian Oct 10 '22 at 17:10

0 Answers0