3

I have drawn a certain Markov chain with a weird transition matrix. Here's the drawing:

https://dl.dropbox.com/u/106018191/markovpolicy.png

And here's the transition matrix:

https://dl.dropbox.com/u/106018191/mathstackex.png

My problem is that I don't quite know how to calculate the steady state probabilities of this chain, if it exists. I need a bit of help, because just getting the $\pi$'s, they don't sum to 1 for me, which is weird.

I've assembled this Markov chain using some data and conditions given to me and, right now, I would insist that it is the inevitable result of a very long process of solving. But now I hit a brick wall and can't continue. Please help?

Edit:

Here's my solution to the problem, using the ordinary ways of calculating steady states.

solution

As you can see, it's a little bit problematic. The probabilities don't add up to 1. I've turned it upside down several times. Is it really just something about my arithmetic, or is this simply a weird Markov chain?

vonbrand
  • 27,812
markovchain
  • 677

1 Answers1

1

There are two equations here: $\pi P=\pi$ and $\sum_i \pi_i=1$. So you will get eight equations to solve for seven variables. You need to omit one of the equations you get from $\pi P=\pi$ and solve the linear equations.

$$P=\left[\begin{array}{ccccccc}l&m&n&0&0&0&0\\o&p&0&0&0&0&0\\ o&p&0&0&0&0&0\\1&0&0&0&0&0&0\\1&0&0&0&0&0&0\\1&0&0&0&0&0&0\\1&0&0&0&0&0&0\end{array}\right]$$

These are the linear equations you need to solve:

$$l\pi_0+o(\pi_1+\pi_2)+\pi_3+\pi_4+\pi_5+\pi_6=\pi_0\\ m\pi_0+p(\pi_1+\pi_2)=\pi_1\\ n\pi_0=\pi_2\\ \pi_0+\pi_1+\pi_2+\pi_3+\pi_4+\pi_5+\pi_6=1$$

Bravo
  • 4,413
  • Hi Shyam. You're right. I did those calculations and will post the resulting image on the OP right now. I basically have a problem because the pi's don't add up to 1. – markovchain Mar 03 '13 at 21:33
  • @user64834: The $\pi$s won't naturally add up to 1 - you have to specify that as a constraint while solving. – Bravo Mar 03 '13 at 21:39
  • Thanks. I got the answer. :) It's 5am, must be the lack of sleep. – markovchain Mar 03 '13 at 21:45