0

Consider a Markov chain with transition probability matrix $P$ given by

$$\displaystyle{P=\begin{bmatrix} \frac{1}{2} & \frac{1}{2} & 0 \\ 0 & \frac{1}{2} & \frac{1}{2} \\ \frac{1}{3} & \frac{1}{3} & \frac{1}{3} \end{bmatrix}}$$

For any two states $i$ and $j$, let $p_{ij}^{(n)}$ denote the $n$-step transition probability of going from $i$ to $j$. Identify the correct statements :

$(a)$ $\displaystyle{\lim_{n\to \infty} p_{11}^{(n)}=\frac{2}{9}}$

$(b)$ $\displaystyle{\lim_{n\to \infty} p_{21}^{(n)}=0}$

$(c)$ $\displaystyle{\lim_{n\to \infty} p_{32}^{(n)}=\frac{1}{3}}$

$(d)$ $\displaystyle{\lim_{n\to \infty} p_{13}^{(n)}=\frac{1}{3}}$

Obviously one approach would be multiplication of the matrix with itself $n$-times to get $P^n$ and then taking limit for its following entries : $P_{11}, P_{21}, P_{32}, P_{13}$. But I think this approach is too much cumbersome for a problem solving approach. Is there some other method to deal with such problems?

am_11235...
  • 2,142

1 Answers1

1

As the graph of state-transitions is connected, irreducible, the rows of the infinity matrix will all be the same left eigenvector of the transition matrix to the eigenvalue 1.

Thus the eigensystem to solve is $$ -\frac12x_1+\frac13x_3=0\\ \frac12x_1-\frac12x_2+\frac13x_3=0\\ \frac12x_2-\frac23x_3=0\\ $$ giving a left eigenvector as $(2,4,3)$ and the normalized solution as $x=\frac19(2,4,3)=(\frac29,\frac49,\frac13)$. This gives two correct and two wrong statements.

Lutz Lehmann
  • 126,666