2

Consider a discrete time Markov Chain on countable state space $X_{0},X_{1},\ldots$. Assume that the chain satisfies the Foster Lyapunov criteria, and since it is countable state space chain, we conclude that it is positive recurrent. Will it still be true that for a bounded function $f$ we have $\frac{1}{n}\sum_{i=0}^{n}f(X_{i})\to Ef(X)$, where the expectation on rhs is wrt the stationary distribution of the process.

user24367
  • 1,286

1 Answers1

1

Yes, when I am thinking about this I dont know why not. As long as is ensured that every state can be expected to be returned to. Then all states in such an irreducible Markov chain being ergodic; follows the chain to be ergodic. With criteria of Foster Lyapunov fulfilled I shall understand that stability is ensured.

Of course, we are talking about a first order Markov chain of no memory.

al-Hwarizmi
  • 4,290
  • 2
  • 19
  • 36
  • Note that I am assuming that I do not know the irreducibility measure. In the proof of Lyapunov drift implies positive recurrence there should be an explicit expression for the irreducibility measure. I want to verify that. I guess I can take the unique stationary measure to be the $\psi$ irreducible measure.

    For example, the conclusion would be false for uncountable state space. In short the chain is such that all states are not reachable.

     – user24367 Jul 20 '13 at 16:37
  • AHA! Have a look here complement to my answer. This goes deeper: http://www.sciencedirect.com/science/article/pii/S0304414900000855 – al-Hwarizmi Jul 20 '13 at 16:49
  • Did the approach work? – al-Hwarizmi Jul 21 '13 at 12:25
  • seems the drift condition is sufficient.... – user24367 Jul 27 '13 at 16:50