0

Let $(X_n)_{n\in\mathbb{N}_0}$ be an irreducible and recurrent Markov chain with state space $E$ and transition matrix $P$. For an $i\in E$ let $t(i)$ denote the random variable $t(i)\colon\Omega\to\mathbb{N}\cup\left\{+\infty\right\}, \omega\mapsto\inf\left\{n\geq 1: X_n(\omega)=i\right\}$.

Now it is to show that for any $i,j\in E$ it is $$ \mathbb{P}_j(t(i)<\infty):=\mathbb{P}(t(i)<\infty|X_0=j)=1. $$

In our reading, we had a rather long (and complicated) way to show that which I will add if you want to see it.

But I wonder why the following which I have on my mind is not working:

Because of the irreducibility there is a $m\in\mathbb{N}$ such that $$ p_{ji}^{(m)}:=\mathbb{P}_j(X_m=i)>0. $$ Isn't $\mathbb{P}_j(t(i)<\infty)=\mathbb{P}_j(\exists n\in\mathbb{N}: X_n=i)$?

And so because the above $m$ exists the probability is 1?

In particular, I do not see where recurrence is needed.

Salamo
  • 1,094
  • How did you get from $\mathbb{P}_j(X_m=i)>0$ to $\mathbb{P}_j(\exists n\in\mathbb{N}: X_n=i)=1$? – David K Feb 09 '15 at 14:08
  • I thought that because for $m\in\mathbb{N}$ it is $X_m=i$ (when starting in $j$) the probability that, starting in $j$, there exists a $n\in\mathbb{N}$ such that $X_n=i$ is 1. – Salamo Feb 09 '15 at 14:10
  • 1
    You have an event with non-zero probability. You even have an infinite number of such events. But it is not always true that the union of an infinite number of events of positive probability is an event of probability $1$. – David K Feb 09 '15 at 14:21
  • So I do missunderstand the event $\left{\exists n\in\mathbb{N}: X_n=i\right}$? It is not enough to find one or infinitely many natural numbers who fullfill that in order to say that this event has probability 1? – Salamo Feb 09 '15 at 14:26
  • Because there can be other natural numbers k, different from $m$ or multiples of $m$, for which $X_k=i$ I cannot say by $\mathbb{P}_j(X_m=i)>0$ that $\mathbb{P}_j(t(i)<\infty)=1$? So I missundestood the $\exists$ as "it is enough to find one such $n\in\mathbb{N}$"? – Salamo Feb 09 '15 at 14:39
  • 1
    There are infinite processes (not this one, of course, because the theorem is true) that have probability $p<1$ to visit state $i$ an infinite number of times, and probability $1-p>0$ that they never visit state $i$. The complex argument shows that this Markov chain is not such a process. – David K Feb 09 '15 at 14:43
  • I think the source of my problem is a wrong understanding of the event $\left{\exists~n\in\mathbb{N}: X_n=i\right}$. I did understand it in the way that it is enough to find one $n$ with this property and then the event has probability 1. But it is meant diferently, right? How would you describe this event? It is the union of ALL $n\in\mathbb{N}$ with that property right? And therefore knowing that $m$ has this property is not enough to know that the event has probability 1, right? – Salamo Feb 09 '15 at 14:46
  • 1
    Ok, I think I got it... it is only $A_m:=\left{X_m=i\right}\subset \left{\exists n\in\mathbb{N}: X_n=i\right}=:B$... and so to know that $A_m$ has positive probability does say nothing about whether the probability of $B$ is 1... so it's simply not helpful what I did... – Salamo Feb 09 '15 at 15:01

0 Answers0