-1

I am trying to answer some questions related to the following markov chain: matrix.The questions are:

  1. Is it irreducible?

  2. What's the stationary distribution?

  3. And I need to find this and explain why I get this answer.

    My answers are :

  4. Yes it is because you can't reach all states e.g. from state 1.

  5. stationary distribution => [1-(5/3)*p, (1/4)*p, p, 1-(5/3)*p]

    The problem that I face is that I am not quite sure how am I supposed to calculate the 3rd one. Any suggestions?

Community
  • 1
cru3lgenius
  • 19
  • "Yes it is because you can't reach all states e.g. from state 1."

    This is false. You can only get to state $4$ from state $1$, and from state $4$, you can only get back to state $1$. So you cannot reach state $2$ from state $1$.

     – 5xum Sep 25 '17 at 11:33
  • @5xum The rationale was correct, the vocabulary was incorrect. – Ian Sep 25 '17 at 11:34

1 Answers1

1

It is not irreducible because of what you said. More specifically, it is "reducible" in the sense that it can be divided into two non-communicating Markov chains. Each of these is irreducible and thus has a unique stationary distribution. The long term distribution will be some combination of these two; can you figure out what combination it is? Hint: it depends on the initial distribution.

Ian
  • 101,645
  • My guess would be that multiplying the probability of being in state 1 for the initial distribution (P(Xo = 1)) with the 3rd entry in the stationary distribution would result in the first probability we are searching for and multiplying again P(Xo = 1) with the 4th entry in the stationary distribution will result in the second probability we are searching for. That would be my guess but I am not sure if it's correct. Can you please elaborate on what do you mean by "The long term distribution will be some combination of these"? – cru3lgenius Sep 25 '17 at 12:24
  • @cru3lgenius You're close. Think about it a bit more carefully: you can divide the chain up into two completely parallel chains on the states ${ 1,4 }$ and ${ 2,3 }$. Each of those, separately, has a unique stationary distribution which is approached in the long run. What does that mean if the initial distribution of the chain is confined to ${ 1,4 }$? What if it's confined to ${ 2,3 }$? What if it is not confined to either? – Ian Sep 25 '17 at 12:29
  • I would say that if we start in either state 1 or 4 the stationary distribution of the chain is going to approach their ({1,4}) stationary distribution (states 2,3 will have 0's for their entries because not reachable) and the same is true for state 2 and 3. If the chain is not confined to either we have to take (P(Xo=1) + P(Xo=4) ) and multiply it with their stationary distribution.The 2 entries we get we gonna put in the stationary distribution of the chain in the 1st and 4th entry respectively. We do the same for states 2 and 3 and that would give us the answer I believe ? – cru3lgenius Sep 25 '17 at 12:59
  • @cru3lgenius You have it right now. – Ian Sep 25 '17 at 13:21