2

Looking at the HMC $$\begin{bmatrix} 1-\alpha & \alpha \\ 0 & 1 \end{bmatrix} $$

How do I prove that the state 2 is recurrent and that state 1 is transient? What does it actually mean by state 1 and state 2?

Thanks for any guidance

Michael Hardy
  • 1
Sarah Jayne
  • 563
  • 2
  • 17

1 Answers1

3

State 1 and state 2 can be anything, for example state 1 = day is sunny and state 2 = day is rainy. Then your matrix tells you the probability of going from sunny to rainy is $\alpha$.

As there is only two states it is enough to show state 1 is transient, as then necessarily the system will always be in state 2. By definition a state is transient if there is a positive probability that the system will leave that state and never return. By looking at your matrix, we see that if the system is in state 2 (a rainy day), the probability of it being a rainy day tomorrow is 1, i.e. guarenteed. Also there is a positive probability of leaving state 1. Hence the result.

Michael
  • 517
  • Oh I understand I think, so the reason that state 1 is transient is because once it leaves, it can never go back? And with state 2 it can leave and go back kind of as it wants? – Sarah Jayne May 22 '14 at 18:06
  • Correct for the first, but for state 2 it is because once you are in state 2, you will never leave, i.e. P(Tomorrow being rainy| yesterday is rainy)=1. Inductively once it rains, it will rain for all time. – Michael May 22 '14 at 18:10
  • Okay thank you so much for clearing that up for me, I really appreciate the help! – Sarah Jayne May 22 '14 at 18:11
  • Technically, the OP did not insist that $\alpha>0$ :) –  May 22 '14 at 18:44