1) In the Markov chain linked below, in the long run, what fraction of the time does the chain spend in state 3?
https://i.stack.imgur.com/tntbj.png
2) Find the probabilities of states 1 , 2 , and 4 in the stationary distribution of the Markov chain 's' linked below. The label to the left of an arrow gives the corresponding transition probability.
1) $P = \begin{pmatrix} 0.5 & 0.5 & 0 & 0 & 0 & 0 \\ 0.5 & 0 & 0.5 & 0 & 0 & 0 \\ 0 & 0 & 0.5 & 0.3 & 0.2 & 0 \\ 0 & 0 & 0.3 & 0 & 0 & 0.7 \\ 0 & 0 & 0.2 & 0 & 0.8 & 0 \\ 0 & 0 & 0 & 0.7 & 0 & 0.3 \end{pmatrix}$
So the long run fraction corresponds to the probability distribution $\pi$ such that: $P^T \pi = \pi $. So we need to find eigenvector of $P^T$ corresponding to eigenvalue = 1. Such eigenvector is $\pi = (0, 0, 0.25, 0.25, 0.25, 0.25)$ (One can check this by checking that $P^T \pi = \pi$). So in the long run 25 percent of time chain spends in state 3.
2)
$P = \begin{pmatrix} 0.2 & 0.4 & 0.4 & 0 & 0 & 0 & 0 \\ 0.2 & 0 & 0 & 0.4 & 0.4 & 0 & 0 \\ 0.2 & 0 & 0 & 0 & 0 & 0.4 & 0.4 \\ 0 & 0.2 & 0 & 0.8 & 0 & 0 & 0 \\ 0 & 0.2 & 0 & 0 & 0.8 & 0 & 0 \\ 0 & 0 & 0.2 & 0 & 0 & 0.8 & 0 \\ 0 & 0 & 0.2 & 0 & 0 & 0 & 0.8 \end{pmatrix}$
For this porblem stationary distribution (eigenvector for eigenvalue = 1) is $\pi = (0.04761905, 0.0952381 , 0.0952381 , 0.19047619, 0.19047619, 0.19047619, 0.19047619) \ $ (Computed numerically)
