0

I am struggling with finding the stationary distribution(s) for a discrete Markov chain with the following transition probability matrix

\begin{bmatrix}1/3&2/3&0&0&0\\ 1/2&1/2&0&0&0\\ 0&1/2&0&1/2&0\\ 0&0&0&1/4&3/4\\ 0&0&0&1/3&2/3\end{bmatrix}

Since this matrix is singular, it is not possible to determine $(I-P)^{-1}$. This I already tried, I also suspect that their might be more than one stationary distribution.

does anybody have an idea?

Jonathan Kiersch
  • 365
  • The third position clearly is not part of any stationary distribution. Using the top left you can find a stationary distribution for the first two positions; using the bottom right you can find a stationary distribution for the last two positions. Any overall stationary distribution will be a convex combination of these two – Henry Jan 25 '19 at 08:54

2 Answers2

0

You can find stationary distributions by solving the equations $xP=x$ for the row vector $x$ where $P$ is th given stochastic matrix. Indeed there are multiple solutions in this case. $x=(t,\frac 4 3 t,0,s, \frac 9 4 s)$ is a solution for any positive $t$ and $s$ and we have to normalize this by the condition $\sum x_i=1$.

Kavi Rama Murthy
  • 311,013
0

Let a row vector $X=(x_1,x_2,x_3,x_4,x_5)$ be a stationary distribution. Then X=XP holds. Solving it directly, we deduce that there exists a real number s and t such that $x_1=3s, x_2=4s, x_3=0, x_4=4t, x_5=9t$. Together with the condition that $x$ is stochastic, we have $7s+13t=1$. Note that this means if you have a five dimensional row vector vector v then $vP^n$ converges to a vector $(3s,4s,0,4t,9t)$ for some real number $s$ and $t$. The values of $s$ and $t$ are determined by the initial condition.

NotoriousJuanG
  • 84