0

I have one exercise about state of Markov chain. The Markov chain is as shown below:

enter image description here

The answer shows B and F are transient as they could reach to absorbing state. But how can I tell, for example, state C is not transient? As the definition of my text book says "state i is said to be transient if, after leaving state i, the probability that it is ever in state i again < 1.". So how I gonna calculate the probability that after C goes to D or E, it is ever come back to state C again is >=1? Thank you

Brian Tung
  • 34,160
whoisit
  • 811
  • Once you are shuffling between $D$ and $E$, on each step there is a non-zero probability (bounded away from $0$) to return to $C$ - hence $C$ is recurrent. – A.S. Dec 11 '15 at 06:33
  • but the the probability from D or E return C is < [email protected]. – whoisit Dec 11 '15 at 06:37
  • On a single step - yes. But eventually the probability of not-return goes down to zero. – A.S. Dec 11 '15 at 06:38
  • @A.S. So what can a state to be called transient? – whoisit Dec 11 '15 at 06:44
  • If the probability of never returning is non-zero, the state is transient. Note that in A.S.'s criterion ("on each step there is a non-zero probability (bounded away from $0$) to return to $C$"), the "bounded away from $0$" part is important. It is not difficult to construct a Markov chain where there is always a non-zero probability of returning to a state, but the state is nevertheless transient. – Brian Tung Dec 11 '15 at 06:58
  • @BrianTung How about state E in the picture? I could make it transient by: firstly E changes to C, then C to D, D to C repeatly forever(though the probablity is very small but it could happen as non zero) ,so it can never go back to E, but how E is not transient? – whoisit Dec 11 '15 at 08:12
  • That infinite sequence has zero probability. It is $0.2 \times 0.3 \times 0.2 \times 0.3 \times \cdots$. – Brian Tung Dec 11 '15 at 08:28

0 Answers0