1

I have been given a theorem stating an analogue of the strong law of large numbers for Markov chains. It states that if $X=(X_n)_{n\in\mathbb{N}}$ is a Markov chain with transition matrix $p$ and $\pi$ is its invariant probability and $f:E\to\mathbb{R}$ is a function integrable with respect to $\pi$, then setting $Y_n:=\frac{X_1+\dotso+X_n}{n}$ one gets: $$\lim_{n\to\infty}Y_n=\sum_{i\in E}f(i)\pi_i=E_{\pi}[f(X_1)],$$ that is, this limit is the integral of $f(X_1)$ with respect to $\pi$. I looked into a couple of references and was unable to find a proof of this, and looking for it on the web is basically impossible because I don't know how to word this in such a way that the query has any efficiency whatsoever. Can someone point me to a reference (not Google Books please, the probability of me not being able to see the page of the proof is close to 1) or post a proof of the theorem here?

MickG
  • 8,645
  • Why this instant downvote? – MickG Jan 25 '15 at 18:27
  • 1
    I think you have typo, that last expectation should be in the denominator. This is also known as the ergodic theorem for Markov chains. Here is one source: http://www.math.uchicago.edu/~may/VIGRE/VIGRE2007/REUPapers/FINALFULL/Casarotto.pdf – Alex R. Jan 26 '15 at 06:31
  • I believe you mistook my theorem for a more general theorem of which mine is a corollary: see here. My theorem is the final corollary, not the Ergodic Theorem. +1 anyway for linking it to the E.T. and helping me find this reference :). And no typo, as the reference shows the expectation is in its rightful place in the numerator :). – MickG Jan 26 '15 at 10:29
  • It is curious that the corollary in question seems to have a hypothesis of aperiodicity while the theorem given to me in class didn't even assume that we students knew what "aperiodic" meant :). – MickG Jan 26 '15 at 10:30

1 Answers1

1

Quite late, but convergence in probability for finite (maybe countable?) state-space is shown in James Norris's book Markov Chains, Section 1.10.

First, the ergodic theorem mentioned by @AlexR is established. This is then extended to the law of large numbers requested by the OP.

Sam OT
  • 6,675