ergodicity in Markov Chains

Question

I have a question regarding ergodicity in the context of Markov chains. I do understand that the intuition about it is that if it’s possible to get from any state to any other state you’ve got an ergodic chain, fine.

My lecturer (who uses Ljungqvist and Sargent’s ‘recursive macroeconomic theory’) handed this algorithm to verify whether a MC is ergodic down to us:

a) find the stationary distribution b) find the invariant function(s) c) check that all the invariant function(s) are constant under the stationary distribution -if so, you can say that the MC is ergodic.

I have a problem with c) – this is probably due to the fact that I’m not a mathematician, but I don’t understand how one goes about that.

He supplied an example- if someone can explain to me, as if I were a child (which, mathematically, I am), how c) is fulfilled there, I would be very grateful.

The transition matrix is (0.8 0.2 0; 0.1 0.9 0; 0 0 1)

It’s straightforward to verify that the stationary distribution, call it π’, is: [1/3γ(1), 2/3γ(2), 1-γ] for any γ∈ [0,1]. It’s equally simple to work out that the invariant functions work out to be: y(1) = y(2), y(3) any

I’m then told that if γ=0, γ=1, that the Markov chain is ergodic, whilst for γ∈ (0,1), the Markov chain is not ergodic. How does that work?

Answer 1

This chain has two non-communicating sets of states: $A=\{1,2\}$ and $B=\{3\}$. A path that visits $A$ can never visit $B$; similarly one that visits $B$ can never visit $A$. A stationary distribution in effect allots some probability to path sequences that stay in $A$ and path sequences that stay in $B$; these are your $\gamma$ and $1-\gamma$. The invariant functions are linear combinations of $\chi_A$ and $\chi_B$, the indicator or characteristic functions of the sets $A$ and $B$. The condition for ergodicity is that $\chi_A(x)$ and $\chi_B(x)$ are constant random variables when $x$ is distributed according to $\pi$. But $\chi_A(x)=1$ with probability $\gamma=\pi(A)$ and $\chi_B(x)=1$ with probability $1-\gamma=\pi(B)$. So unless $\gamma=0$ or $=1$, the random variables $\chi_A(x)$ and $\chi_B(x)$ both take on both possible values $0$ and $1$ with positive probabilities.

A slogan worth remembering is: a stationary process breaks into the union of non-communicating ergodic parts. A stationary function will be constant on the ergodic parts. The process itself is ergodic only if only one piece gets non-zero probability, that is, the stationary functions are almost surely constant.