2

In Norris: Markov Chains the closed class C is defined as one for which $i\in C$ and $P_i(X_n=j \text{ for some }n\ge0)>0$ implies that $j\in C$.

Here's theorem 1.5.6 from the book with proof

Every finite closed class is recurrent.

Proof: Suppose C is closed and finite and that the chain $X_n$ starts in $C$. Then for some $i\in C$ we have

$$0<P(X_n =i\text{ for infinitely many }n)=P(X_n =i \text{ for some } n)P_i(X_n =i\text{ for infinitely many }n)$$

by the strong Markov property. So $i$ is not transient, hence $C$ is recurrent.

I'm confused because take for example a state the chain never reaches, say $y$. Then the union of an irreducible set (where all pairs of states communicate) and $y$ is not irreducible anymore but it is closed. So $y$ cannot be recurrent since it is never reached. Hence the irreducibility condition might be needed in addition closed + finiteness to imply recurrence?

adamG
  • 665
  • 1
  • 10
  • 23

2 Answers2

3

The word "class" as defined in the book means "comunicating class" (i.e. class = irreducible set). Since you have added the state $y$ it is not anymore comunnicating since it is unreachable and thus the property does not apply.

Bunder
  • 2,423
1

In your example, provided that $y$ itself belongs to a closed class, then $y$ is recurrent and is hit infinitely often starting at $y$. That is, $$\mathbb{P}_y(X_n=y\mbox{ for infinitely many }n)>0.$$ Be careful, the definition of a recurrent state depends on the starting point, and it certainly doesn't guarantee that we hit state $y$ infinitely often starting somewhere else.

  • Thanks very much! How does the problem change if I change the definition of a closed set to $i\in C$ and $P_i(T_j<\infty)>0$, where $T_j$ is the hitting time of j, then $j \in C$? In the first definition (above) n (that is the step when y is reached) can be infinity, can't it? However I think that in the second definition my counterexample does apply and irreducibility is need separately. Am I right? – adamG Oct 29 '13 at 09:18
  • You are describing a communicating class, not a closed class. So now the original class plus the new state $y$ is no longer a class. But it is still recurrent. Recurrence has nothing to do with whether you can go from one state to another. –  Oct 29 '13 at 12:31