0

Consider a Markov chain with the states 0,1,2,3,4,5,6 and transition matrix $$ P=\begin{pmatrix}\frac{1}{5} & \frac{3}{5}& 0 & 0 & \frac{1}{5} & 0 & 0\\0 & 0 &1 & 0 &0&0&0\\0&\frac{1}{3}&0&\frac{2} {3}&0&0&0\\0&1&0&0&0&5&0\\0&0&0&0&0&1&0\\0&0&0&0&0&0&1\\0&0&0&0&1&0&0\end{pmatrix} $$

Show that $$ (p_{04}^{(n)})_{n\in\mathbb{N}} $$ does not converge as $ n\to\infty$. Determine limsup and liminf of the sequence.

I am wondering why the sequence does not converge because as far as I see it is $$ p_{04}^{(n)}=\left(\frac{1}{5}\right)^n. $$

which goes to 0 as $ n\to\infty$.

mathfemi
  • 2,631
  • 1
    This is not $p_{04}^{(n)}$. For example one can go from $0$ to $4$ in four steps in two ways, either as $0\to0\to0\to0\to4$ or as $0\to4\to5\to6\to4$, hence $$p_{04}^{(4)}=\left(\frac15\right)^4+\frac15\ne\left(\frac15\right)^4.$$ – Did Oct 25 '14 at 10:01
  • 1
    Are you sure about the exercise? Showing that $(p^{(n)}{44})$ has no limit is trivial, but I fail to see a simple argument for $(p^{(n)}{04})$. – Did Oct 25 '14 at 10:09
  • This is exactly the problem: In our first reading it was said that this has to be done by Perron-Frobenius-Theorem and it was said that this theorem will be given later in the lecture. But now this exercise appears on the first work sheet and we did not have the theorem yet of course. – mathfemi Oct 25 '14 at 10:17
  • One expects that $p^{(n)}_{04}$ does not converge because the chain sooner or later enters the cycle $4\to5\to6\to4$... but it could happen that the time and place the chain is entering the cycle are "carefully" distributed so that the cyclic behaviour is evened out. To get rid of this (remote) possibility is not obvious, anyway Perron-Frobenius will not suffice. Close the question? – Did Oct 25 '14 at 10:23

1 Answers1

1

Hint: here the notation is:

$$ p^{(n)} = n\text{th power of }p\\ p^{(n)}_{04} = (n\text{th power of }p)_{04} $$

mookid
  • 28,236