2

I have the following Markov Chain
\begin{bmatrix} 0 & 1 & 0 & 0 \\ 0 & \frac{1}{3} & \frac{2}{3} & 0 \\ 0 & 0 & 0 & 1 \\ \frac{3}{5} & 0 & 0 & \frac{2}{5}\\ \end{bmatrix} Now the question is. Let T be the time to, for the first time, arrive in state 1, after leaving from state 1. Calculate the expectation and variance of T. The first part I did, calculcating recursively led me to a mean first return time of $\frac{31}{6}$. However I cannot figure out how to calculate the variance.

Edit:
The first part I did like this.
$u_{11}$ = 1 + $u_{21}$
$u_{21}$ = 1 + $\frac{1}{3}u_{21}$ + $\frac{2}{3}u_{31}$
$u_{31}$ = 1 + $u_{41}$
$u_{41}$ = 1 + $\frac{2}{5}u_{41}$
and solving gives me $u_{11}$ = $\frac{31}{6}$.

Jasper
  • 145
  • Could you use the same kind of recursive computation to find the expected value of the first return time squared? – Brian Borchers Oct 08 '17 at 00:07
  • You mean like this: $u_{11}^2$ = $(1+u_{21})^2$? – Jasper Oct 08 '17 at 12:50
  • You need to be careful to distinguish between random variables and their expected values. It's not true in general that $E[X^{2}]=E[X]^{2}$. You seem to be a bit confused by what exactly $u_{11}$ is. – Brian Borchers Oct 08 '17 at 12:59
  • Well I thought that $u_{11}$ is the expected number of steps it takes to arrive at state 1, departing from state 1. So $u_{11}$ = E[X] with X the number of steps needed to go from state 1 to state 1. Now I know that Var[X] = E[$X^2$] - $E[X]^2$. So what I need to know is how to calculate E[$X^2$] .. – Jasper Oct 08 '17 at 13:09
  • Can you explain to me how to go about this? – Jasper Oct 10 '17 at 15:58

0 Answers0