0

My question is: Starting in state 1 what is the probability on ending up in state 3 resp. state 4?

The transition matrix T= \begin{bmatrix} 0& 0.8& 0&0.2\\ 0&0.2&0.6&0.2\\ 0&0&0&0\\ 0&0&0&0 \end{bmatrix} Both state 3 and 4 are absorbing states, and once you have left state 1 you can not get back there. At first I calculated it to be 0.6 chance to end up in state 3 and 0.4 for state 4. I got this using

$\pi_0=0.8\pi_1+0.2\pi_3$,

$ \pi_1=0.2\pi_1+0.6\pi_2+0.2\pi_3$,

$\pi_2=\pi_2$,

$\pi_3=\pi_3$

$\pi_0+\pi_1+\pi_2+\pi_3=1$

Setting $\pi_0=0$ as is not possible to end up there.

However, recalculating it I get something looking very wrong $47/20\pi_2+29/20\pi_3=1$

I also would like to know how many steps on average does it take to get to state 3?

S.n
  • 145
  • Your states seem to be shifted by 1 (1-4 versus 0-3). What is the definition of $\pi_i$, and how do you get those equations? – Michael Nov 30 '16 at 12:48
  • I cannot come up with a definition of $\pi_i$ under which those equations make sense. I suspect you are incorrectly "combining" equations for steady state and equations for recursively computing the probability of ending in state 3. If you have states ${1, 2, 3, 4}$ and you define $A_i$ as the event of ending in state 3 (given you start in state $i$), you can compute $P[A_i]$ in terms of $P[A_j]$ for $j \in {1, 2, 3, 4}$ using the law of total probability. Of course, your transition matrix makes no sense as the last two rows are all zero. – Michael Nov 30 '16 at 12:56
  • [T=\begin{bmatrix} 0& 0.8& 0&0.2\ 0.2&0&0.6&0.2\ 0&0&1&0\ 0&0&0&1 \end{bmatrix}] – S.n Nov 30 '16 at 13:20
  • I'm sorry, I put in the wrong matrix, of course the two last rows contains the absorbing states. – S.n Nov 30 '16 at 13:22
  • I just watched a video on absorbing Markov chains (PatricJMT), where you calculate a limiting matrix. However I get the answer to be negative, which can't be correct – S.n Nov 30 '16 at 13:24
  • If you could define $\pi_i$ it would help tremendously. Equations cannot be understood unless the variables are clearly defined. Right now, the only equations that can be verified as "true" are $\pi_2=\pi_2$ and $\pi_3=\pi_3$. – Michael Nov 30 '16 at 23:18

0 Answers0