1

Consider an irreducible Markov chain with an invariant distribution $\pi$. Then show that if $\pi(x)>0$ for some $x \in S$, where $S$ is the state space, then $x$ is recurrent.

Here's what I was trying: so we know that $P_x(T_y<\infty)>0$ for any $x,y\in S $ by irreducibility and that $\pi(x)=\sum_{y \in S}\pi(y)p_{xy} = \lim_{n \rightarrow \infty} p_{zx}^{(n)}>0$.

We want to show that $P_x(T_x<\infty)=1$ or equivalently that $E_x(N(x))=\infty$ where $N(x)$ is the number of returns to $x$.

Could anyone give me a hint as I'm running into too many dead ends?

adamG
  • 665
  • 1
  • 10
  • 23

1 Answers1

2

If the expected number of returns to $x$ is finite, then because $$E_x(N(x))=E_x\left(\sum_{n=0}^\infty 1_{[X_n=x]}\right)=\sum_{n=0}^\infty p^{(n)}_{x,x}<\infty,$$ we have $ p^{(n)}_{x,x}\to0$. In this case the Cesàro averages also go to zero, so $$\pi(x)=\lim_{N\to\infty}{1\over N}\sum_{n=1}^N p^{(n)}_{x,x}=0.$$

That is, the invariant measure $\pi$ puts no weight on a transient state.

Note that your equation $\pi(x)= \lim_{n \rightarrow \infty} p_{zx}^{(n)}$ is not true unless the chain is aperiodic.