1

I'm a bit confused by this theorem, not sure of the name in english but here are the details:

Let the markov chain $X = (X_n)_{n\geq 0}$ with transition matrix $\mathbb{P}$ and state space $S$.

Let $X$ be irreducible then, if a state in $S$ is transient, then all states are transient. And every state will only be visited a finite number of times.

I'm not quite sure how a markov chain could satisfy this property. I can see how a subset of states might be transient, but to have a markov chain you need at least one recurrent state, perhaps I'm misunderstanding something.

If such a markov chain exists, then I'd like to see an example of one, I've tried searching but I can't seem to find anything on this.

Any help would be greatly appreciated.

Fainted
  • 181
  • 3
    A Markov chain may have infinitely many states, for example the classical random walk on $\Bbb Z^3$ has every sate transient. – Angina Seng Jun 29 '17 at 06:16

1 Answers1

2

The assumptions that $X$ is irreducible and that there exists a transient state $z$ imply, that $S$ is not finite. The point is the following: Assume there is a recurrent state $x$. Then there is a $m$ such that you reach $z$ in $m$ steps from $x$ with probability $p>0$. So each time you return to $x$, you throw a $p$-coin if you will reach $z$ in $m$ steps.
As an example, onsider the random walk on $\mathbb{Z}$ which goes to the right with probability $0.99$ and left with probability $0.01$. It should be clear, that the chain goes to $\infty$ then.

crankk
  • 1,439
  • It's not clear to me how you would prove that the states of this irreducible chain are transient. Can you explain further? – Elle Najt Jun 29 '17 at 06:26
  • I want to use Borel-Cantelli 2. There is a positive probability, that you reach $z$ from $x$ in $m$ steps. If $x$ is recurrent, there is infinitely often the possibility to do so, and because of the Markov property, indepentently. By BC 2 this will occure a.s. infinitely often, a contradiction to the assumption that $z$ is transient. – crankk Jun 29 '17 at 06:49
  • I mean your example on the integers. – Elle Najt Jun 29 '17 at 07:44
  • This is the strong law of large numbers. You can write your chain as $X_n=Z_1+...+Z_n$, where $Z_i$ are i.i.d with $P(Z_i=1)=0.99,P(Z_i=-1)=0.01$. By the LLL $X_n/n \to E[Z_i]=0.98$, so $ \lim X_n = \lim 0.98n = \infty$. – crankk Jun 29 '17 at 08:12
  • It's not clear how that proves that the chain is not recurrent. Even if I drift to infinity, it is plausible that I could visit $0$ infinitely often. – Elle Najt Jun 29 '17 at 08:14
  • I do not say that it drifts to infinity, I say that it converges to infinity almost surely (the strong LLL). So it can visit any point just finitely often. The number of visits in 0 is a random variable which is a.s. finite. – crankk Jun 29 '17 at 08:57
  • crankk Oh I see. Thanks! – Elle Najt Jun 29 '17 at 08:59