0

Consider a Markov's chain on the state space $\{0,1,\ldots\}$ with the transition matrix which is an identity matrix. This gives us that every state communicates only with itself. From the definition of irreducible Markov chain, in an irreducible Markov chain every state must communicate with each other. So in this case the Markov chain is not irreducible.

However, the answer given is "this chain is irreducible and positive recurrent." I don't understand how? Can someone make me understand, please?

rubik
  • 9,344
shwetha
  • 695

1 Answers1

0

Yes, since there there does not exist a path connecting from one node to another node that is there exists two states between which a path is not there, so it is not irreducible as you figured out.

A recurrent state is also known as a Persistent state.

A few notations before proceeding -

$f_{jk}^{(m)} =$ Probability that the given system reaches state $k$ for the first time after $m$ transitions.

A state $j$ is said to be persistent or a recurrent state if $F_{jj} = \sum_{t=1}^{\infty} f_{jj}^{(n)} = 1$ that is starting from state $j$ and returning to state $j$ is certain.

If $F_{jj}<1$, then the state $j$ is called Transient state.

And you see here in your case, $f_{jj}^{(0)} = 1 \forall j = 0,1,2,..,n$

So all the states are recurrent states.

BAYMAX
  • 4,972