Steady-state and Equation System

Question

Two questions:

Given the transition matrix:

$ \begin{vmatrix} \ 0.4 & 0.4 & 0.2 \\ \ 0.5 & 0.3 & 0.2 \\ \ 0.1 & 0.5 & 0.4 \end{vmatrix} $

I would like to know HOW to find the steady-state of this.

I've tried to do that trying to solve the system by hand:

$ \begin{aligned} \ a = 0.4a+0.5b+0.1c \\ \ b = 0.4a+0.3b+0.5c \\ \ c = 0.2a+0.2b+0.4c \\ \ 1 = a+b+c \\ \end{aligned} $

Question 1: Is it possible to solve this system by hand? What is the best way to do that? If it not possible to solve, how do i see this impossibility?

Question 2: Wolfram said to me that this system is impossible (please, answer the first question (:). What this fact means to this Markov chain?

It's impossible that it's impossible. Every finite state space Markov chain has a steady state probability vector. — , Apr 07 '13 at 21:31

score 2 · Answer 1 · edited Apr 13 '17 at 12:21

2

In the steady-state, the vector of probabilities multiplied by the transition matrix has to result in the vector of probabilities. Therefore, you have to look for the Eigenvector of the matrix associated to Eigenvalue 1.

A brute-force solution is to use Excel or pen and paper and multiply some assumed initial vector of probabilities with the matrix. If you repeat that a number of times with the resulting vector as new input, you end up at:

a = 0.363636
b = 0.386363
c = 0.25

A related question/discussion is here.

edited Apr 13 '17 at 12:21

Community

1

answered Apr 07 '13 at 21:55

Axel Kemper

4,943

1

Or as fractions $(a,b,c)=(4/11,17/44,1/4)$. – Apr 07 '13 at 22:06
Please, show how to find these values without using Excel or brute-force. – richardaum Apr 08 '13 at 02:10
There is an online calculator available with some explanations: http://math.plussed.net/markov/ – Axel Kemper Apr 08 '13 at 06:33

Steady-state and Equation System

1 Answers1