0

I am given a matrix with space {0,1,2,3,4}.

I already calculated the invariable probability vector.

However, the question asks to give the expected number of steps: -given Xo=0 to go back to state "0". -given Xo=0 to go back to state "3".

For Xo=0 --> state 0, I just used $$1/pi(0)$$ However to figure out the other question, I think I need to create state 3 as an absorption state, so I rearranged my matrix, with state 3 as the top left corner with probability 1. then I tried to use the theory where (I-Q)inverse * S

$$ \begin{matrix} P & 0 \\ S & Q \\ \end{matrix} $$ then rearranged to $$ \begin{matrix} p(3,3)=1 & 0 \\ S & Q \\ \end{matrix} $$

However, I realized this will only give me that, there is a 100% chance to land back at this spot if I force it to be an absorption state. (I get the column vector [1; 1; 1; 1])

If anyone can help me, I would greatly appreciate it.

The theory is mainly important, but here's the matrix itself to make things clear. I tried reading the textbook, but it is extremely unclear, and offers no more help.

$$ \begin{matrix} 0 & 1/2 & 1/2 & 0 & 0\\ 0 & 0 & 0& 1/5 & 4/5\\ 0 & 0 & 0& 2/5 & 3/5\\ 1 & 0& 0 & 0 & 0\\ 1/2 & 0 &0 & 0 & 1/2\\ \end{matrix} $$

sahimat
  • 532
  • I figured it out, the answer and thinking is this. (I-Q) inverse gives the matrix that adds up the expected times, so if you add up the respective row, you get the expected time from that state, to visit the "absorptive" state. Of course multiplying it by the other matrix should give you 100% chance, otherwise something went wrong. – sahimat Jan 23 '15 at 01:54
  • Should anyone knowledgeable in stochastic processes like to chime in and make things more clear, I'd be happy to give the checkmark to "close" the question. – sahimat Jan 23 '15 at 01:56

0 Answers0