Distribution of a Markov Chain

Question

I'm having a very hard time understanding a theorem in my notes stating the Markov property. It says that for a Markov-chain $(X_n)_n$ with initial distribution $\mu$ and transition matrix $p$ we have:

$\mathbb{P}((X_{m+n})_{n\geq0} \in C||\mathcal{F}_m)=\mathbb{P}((X_{m+n})_{n\geq0} \in C||X_m)=P_{X_m(.)}(C)$

where $P_{x}(C)$ denotes the distribution of a Markov chain with initial distribution $\delta_{x}$ and transition matrix $p$. I have a really hard time understanding why the second equality holds, and I think that's probably because I don't fully understand how the distribution of a Markov proces is supposed to work.

Are the first two expressions supposed to be the same? – Misha Lavrov Mar 25 '21 at 21:23 — Misha Lavrov, Mar 25 '21 at 21:23
Nope, my bad! Apologies – Othman El Hammouchi Mar 25 '21 at 21:25 — Othman El Hammouchi, Mar 25 '21 at 21:25

score 1 · Answer 1 · answered Mar 25 '21 at 21:36

The notation is a bit confusing, but here's what I'm assuming is going on.

The first equation is the actual Markov property. It says that the distribution of "future" states $X_{m+1}, X_{m+2}, X_{m+3}, \dots$ given the "present" state $X_m$ and the "past" states $X_0, X_1, \dots, X_{m-1}$ is the same as the distribution of the future states given only the present state.

The second equation is a time-homogeneity property - usually part of the definition, but not always. It says that the distribution of the future states $X_{m+1}, X_{m+2}, \dots$ given the present state $X_m$ is the same as the distribution we'd get for the entire Markov chain, if it had started at $X_m$.

An example of a Markov chain that does not satisfy the second property is a random walk on the integers that "slows down": on the $n^{\text{th}}$ step, it goes left with probability $\frac1{2n}$, goes right with probability $\frac1{2n}$, and stays put with probability $1 - \frac1n$. This decision does not depend on how it got to the current state, so the Markov property holds. But this Markov chain is not time-homogeneous. If you're in state $0$ at the beginning of time, you'll always end up in state $1$ or $-1$ next. If you're in state $0$ before the $100^{\text{th}}$ step, there's a $99\%$ chance you'll stay at $0$.

So the second equality doesn't always hold? What about the claim that $P_{X_m(.)}(C)$ is $\mathcal{F}$-measurable? Is that true and if so why? Thanks a lot! — Othman El Hammouchi, Mar 25 '21 at 21:44
Neither of these "always hold"; they are assumptions we make about a random process. Some people make only the first assumption when they call something a Markov chain, some people make both assumptions. — Misha Lavrov, Mar 25 '21 at 21:46

Distribution of a Markov Chain

1 Answers1