0

In our reading we had the following example for a Markov chain.

Markov chain

I cite from the reading:

Here we have three communicating classes: $\left\{0\right\}, \left\{1,2,3\right\}$ and $\left\{4,5,6\right\}$. Two classes, namely $\left\{1,2,3\right\}$ and $\left\{4,5,6\right\}$, are closed, meaning once you enter you will not leave anymore.

Okay, I do understand that. But now there is a passage I cannot see.

These two classes ($\left\{1,2,3\right\}$ and $\left\{4,5,6\right\}$) are recurrent. This means that you return again and again to every state.

I cannot verify this passage! Why do we return to state 3 again and again? It is possible that from 3 we get to 1, then to 2, again to 1, to 2, and so on... so that we never get back to 3.

Where is the mistake in my thought? Or maybe the definition of recurrence is not good?

  • The probability to never get to state 3 when starting from 1 or 2 is 0. THis is the definition of a recurrent state. – mookid Oct 14 '14 at 10:27
  • just a matter of precise definition. a state is "recurrent" if its probability of recurring is $1$. this allows for a set of realizations of measure zero in which the recurrent state does not actually recur. – David Holden Oct 14 '14 at 10:28
  • I'm not familiar with this class of 'recurrent' Markov systems, but you could consider a probabilistic argument. The probability you never return to $3$ is $\lim\limits_{n \to \infty} (\frac{1}{3})^n=0$ – G. H. Faust Oct 14 '14 at 10:29

0 Answers0