0

Let $S$ be a countable state space and $(X_n)$ an $S$-valued discrete Markov chain. Then $$ \mathbb P [X_{n+1} = x_{n+1} |X_n = x_n, \ldots, X_0=x_0] = \mathbb P [X_{n+1} = x_{n+1} |X_n = x_n] $$ for all $x_0, \ldots, x_{n+1} \in S$. This is called the Markov property. Inspired by this thread, I have proved below generalization, i.e.,

Theorem $$ \begin{align} & \mathbb P [X_{n+1} \in S_{n+1} |X_{i_M} = x_{i_M}, X_{i_{M-1}} \in S_{i_{M-1}} \ldots, X_{i_m} \in S_{i_{m}}] \\ = &\mathbb P [X_{n+1} \in S_{n+1} |X_{i_M} = x_{i_M}] \end{align} $$ for all $i_m \le i_{m+1} \le \cdots \le i_M \le n$ such that $S_{n+1} \subset S, x_{i_M} \in S, S_{i_{M-1}} \subset S, \ldots, S_{i_m} \subset S$.

Can we generalize further by relacing $X_{i_M} = x_{i_M}$ with $X_{i_M} \in S_{i_{M}}$? If not, is there some intuition why it does not hold?

Thank you so much for your elaboration!

Proof Because $\mathbb P [\bigcup_n A_n | B] = \sum_n \mathbb P [A_n | B]$, it suffices to consider the case $S_{n+1}$ is a singleton, i.e., $S_{n+1} = \{x_{n+1}\}$. Let $\pmb S := S_{i_{M-1}} \times \cdots \times S_{i_{m}}$ and $\pmb Y := (X_{i_{M-1}}, \ldots, X_{i_m})$. Then $$ \begin{align*} & \mathbb P [X_{n+1} = x_{n+1} |X_{i_M} = x_{i_M}, X_{i_{M-1}} \in S_{i_{M-1}} \ldots, X_{i_m} \in S_{i_{m}}] \\ = &\mathbb P [X_{n+1} = x_{n+1} |X_{i_M} = x_{i_M}, \pmb Y \in \pmb S] \\ = & \frac{\mathbb P [X_{n+1} = x_{n+1} ,X_{i_M} = x_{i_M}, \pmb Y \in \pmb S]}{\mathbb P [X_{i_M} = x_{i_M}, \pmb Y \in \pmb S]} \\ = & \frac{\sum_{\pmb y \in \pmb S } \mathbb P [X_{n+1} = x_{n+1} ,X_{i_M} = x_{i_M}, \pmb Y = \pmb y]}{\sum_{\pmb y \in \pmb S }\mathbb P [X_{i_M} = x_{i_M}, \pmb Y = \pmb y]} \\ = & \frac{\sum_{\pmb y \in \pmb S } \mathbb P [X_{n+1} = x_{n+1} |X_{i_M} = x_{i_M}, \pmb Y = \pmb y] \mathbb P [ X_{i_M} = x_{i_M}, \pmb Y = \pmb y]}{\sum_{\pmb y \in \pmb S }\mathbb P [X_{i_M} = x_{i_M}, \pmb Y = \pmb y]} \\ = & \frac{\sum_{\pmb y \in \pmb S } \mathbb P [X_{n+1} = x_{n+1} |X_{i_M} = x_{i_M}] \mathbb P [ X_{i_M} = x_{i_M}, \pmb Y = \pmb y]}{\sum_{\pmb y \in \pmb S }\mathbb P [X_{i_M} = x_{i_M}, \pmb Y = \pmb y]} \\ = & \mathbb P [X_{n+1} = x_{n+1} |X_{i_M} = x_{i_M}], \end{align*} $$ where the fifth inequality follows from the following Lemma, i.e.,

Lemma $$ \mathbb P [X_{n+1} = x_{n+1} |X_{i_M} = x_{i_M}, \ldots, X_{i_m}=x_{i_m}] = \mathbb P [X_{n+1} = x_{n+1} |X_{i_M} = x_{i_M}] $$ for all $i_m \le i_{m+1} \le \cdots \le i_M \le n$ and $x_{i_m}, \ldots, x_{i_M}, x_{n+1} \in S$.

Akira
  • 17,367

1 Answers1

1

The property cannot be generalized further in the manner you propose. For example, in the identity $$ P(X_3=x_3 \mid X_2=x_2, X_1=x_1,X_0=x_0) = P(X_3=x_3\mid X_2=x_2),\tag1 $$ which is a consequence of the Markov property, we cannot replace the event $\{X_2=x_2\}$ with the event $\{X_2\in S\}$. If we perform this replacement, the LHS of (1) becomes $$ P(X_3=x_3\mid X_1=x_1, X_0=x_0)\tag2 $$ while the RHS of (2) becomes $$P(X_3=x_3)\tag3$$ and (2) and (3) are not equal.

What is the intuition here? The Markov property is sometimes described as "conditional on the present, the future is independent of the past". More precisely, the Markov property holds when we condition on a specific value of the "present" variable. (In (1), the "present" variable is $X_2$; in your theorem, it is $X_{i_M}$.) When we replace a specific value of the present variable by a range of values, the Markov property has nothing to say. Indeed, in the pathological case where the range of "present" values equals the entire state space, we have situation (2) where the "present" variable $X_2$ has disappeared. (And if $X_1$ is now to be considered the "present" variable in (2), then the Markov property would assert that (2) equals $P(X_3=x_3\mid X_1=x_1)$.)

grand_chat
  • 38,951