4

I have a Markov chain with state space $E = \{1,2,3,4,5\}$ and transition matrix below:

$$ \begin{bmatrix} 1/2 & 0 & 1/2 & 0 & 0 \\ 1/3 & 2/3 & 0 & 0 & 0 \\ 0 & 1/4 & 1/4 & 1/4 & 1/4 \\ 0 & 0 & 0 & 3/4 & 1/4 \\ 0 & 0 & 0 & 1/5 & 4/5 \end{bmatrix} $$

Given the initial distribution $\pi = (1/2,0,0,1/2,0)$, how would I calculate $\mathbb{P}(X_2 = 4)$? Do I have to multiply $\pi$ by the transition matrix etc.?

Michael Hardy
  • 1
Henry
  • 93

1 Answers1

1

If "initial distribution" means the distribution of $X_1$, then the distribution of $X_2$ is $\pi M$, where $M$ is the transition matrix above.

If "initial distribution" means the distribution of $X_0$, then the distribution of $X_2$ is $\pi M^2$.

Michael Hardy
  • 1
  • Thanks Michael! I think the question means the "initial distribution" means the distribution of $X_0$.

    Just one quick question, I cant seem to multiply $M^2 \pi$, because the dimensions (row and columns) are not correct, but I can multiply $\pi M^2$, and I get the correct answer $(59/160)$. So the distribution $M^2 \pi$ or $\pi M^2$?

     – Henry Apr 10 '12 at 23:41
  • It should be $\pi M^2$. This is a "right stochastic matrix" (rows sum up to 1), which means it's on the "right" of a matrix multiplication. It is my understanding that transition matrices are by convention situated on the right. I'm not sure why this is so; maybe someone else can enlighten us. – Yang Apr 11 '12 at 00:01
  • Yes, please, can someone enlighten us! @X.Yang, so if I have a left stochastic matrix, would it be on the "left" of a matrix multiplication? Or right? – Henry Apr 11 '12 at 00:08
  • On the left - that's where the name comes from. A left stochastic matrix is defined as one where all the columns sum up to 1. This is essential because, when you end up doing the matrix multiplication, you'll see that each state has a total probability 1 of going to any other state - which only makes sense. Same thing for right stochastic matrices. – Yang Apr 11 '12 at 00:15
  • Thanks, what do you mean then when you said that by convention the transition matrices are situation on the right? – Henry Apr 11 '12 at 00:20
  • I mean that when we write down the stochastic matrices, we usually write them down as right stochastic matrices unless otherwise stated. Sorry for the confusion. – Yang Apr 11 '12 at 00:21
  • Thanks for your clarification, and no worries :) – Henry Apr 11 '12 at 01:10
  • I confused the roles of rows and columns, but now I've fixed it. – Michael Hardy Apr 11 '12 at 02:56