1

Let $\{X_n\}$ be a Markov chain on a state-space $E$. A state $i$ is recurrent if $$P(X_n = i\;\text{for some} \;n\geq 1|X_0=i) = 1\tag{Definition 1}$$

$$\text{for some}\; n\geq 1\; P(X_n=i|X_0=i)=1\tag{Definition 2}$$

Definition $2$ is supposed to be incorrect. Could someone provide an example where definition $1$ holds but definition $2$ does not or vice versa? Or otherwise show me why they aren't equivalent. I've been told it might have something to do with when $n$ tends to infinity.

mrnovice
  • 5,773

1 Answers1

0

If $\{X_n\}$ is any Markov chain with finite state space such that $0<p_{ij} <1$ for all $i,j$ then the first condition holds because such a chain is (positive) recurrent. However condition 2) fails because all elements of the n-th power of the transition matrix are all also less than $1$. 2) implies 1) is obvious.

Kavi Rama Murthy
  • 311,013
  • I don't really understand your reasoning for why definition $2$ fails, can you explain it in more simple terms? – mrnovice Oct 16 '18 at 09:28
  • $P{X_n=i|x_0=i}$ is nothing but the $(i,i)$ element of the matrix $P^{n}$. Now $p^{(n)}{ij}=\sum_k p{ik}p^{(n-1)}{ki}$. From this you can easily see by induction that $p^{(n)}{ij}<1$ for all $n$ for all $i,j$. In particular $P{X_n=i|x_0=i}=p^{(n)}_{ij}<1$. – Kavi Rama Murthy Oct 16 '18 at 09:35
  • Okay, but then why doesn't definition $1$ also fail in your reasoning? It will never equal to $1$ for finite $n$ – mrnovice Oct 16 '18 at 09:42
  • You cannot express $P{X_n=i , for, some, n |X_0=i}$ in terms of the transition probabilities $p_{ij}$. – Kavi Rama Murthy Oct 16 '18 at 09:44