2

Let $X_n$ be a DTMC, with transition matrix P and state-space I. Let $Y_m=X_{T_m}$ for $m \in \mathbb{N}$.

Define $T_0=\inf\{n\geq0:X_n\in J\subset I\}$ and $T_{m+1}=\inf\{n> T_{m}:X_n\in J\subset I\}$. These are stopping times, so we can use the strong markov property when conditioning on them. I'm having a bit of trouble understanding a passage from a book.

It goes like this: For $i_0,...,i_{m+1} \in J$ we have $$P(Y_{m+1}=i_{m+1}|Y_0=i_0, ..., Y_m=i_m)=P(X_{T_{m+1}}=i_{m+1}|X_{T_{0}}=i_0,..., X_{T_{m}}=i_m)=P_{i_m}(X_{T_{1}}=i_{m+1})=h^{i_{m+1}}_{i_m}$$ where $h^{i_{m+1}}_{i_m}=P(\inf\{n\geq0:X_n=i_{m+1}\}<\infty|X_0=i_m)$

I don't understand the last two equalities...

I would think that $P(X_{T_{m+1}}=i_{m+1}|X_{T_{m}}=i_m)=P(X_{T_{m+1}-T_m}=i_{m+1}|X_{0}=i_m)$.

Any help would be appreciated.

An old man in the sea.
  • 5,241

1 Answers1

1

\begin{align} P(X_{T_{m+1}}=i_{m+1}|X_{T_{0}}=i_0,..., X_{T_{m}}=i_m) &= P(X_{T_{m+1}}=i_{m+1}|X_{T_{m}}=i_m) \\ & \qquad\qquad\qquad\text{it might help to expand $T_{m+1}$ here:} \\ &= P(X_{\inf\{n> T_{m}:X_n\in J\subset I\}}=i_{m+1}|X_{T_{m}}=i_m) \\ &= P(X_{\inf\{n> T_{0}:X_n\in J\subset I\}}=i_{m+1}|X_{T_{0}}=i_m) \\ &= P(X_{T_{1}}=i_{m+1}|X_{T_{0}}=i_m) \\ &= P_{i_m}(X_{T_{1}}=i_{m+1}) \\ &= h^{i_{m+1}}_{i_m}. \end{align}

The probability $P(X_{T_{m+1}-T_m}=i_{m+1}|X_{0}=i_m)$ isn't right because $X_{T_{m+1}-T_m}$ is interpreted as: take the difference of the stopping times $T_{m+1}$ and $T_m$, then advance that number of steps starting at the beginning (step $0$) and take the value of $X$ at that point. The value of $X_{T_{m+1}-T_m}$ therefore could be anything in the state space, not necessarily in set $J$.

Mick A
  • 10,208
  • Mick, I'm very grateful for this answer. However, since I'm new to the subject, and the book I'm using doesn't really explain in detail some things, I'm unsure regarding some steps in your answer. The first equality, how is it proved just by the Strong Markov property? Aren't we assuming in that equality that $T_{m+1}<\infty$, because the strong markov is for $X_{T_m+n}$, where $n$ is deterministic? Also, in the last equality, could you elaborate a bit? I don't understand why it's true. – An old man in the sea. Dec 14 '15 at 21:12
  • @Anoldmaninthesea. Assuming that $T_{m+1}<\infty$? That's built into the event $X_{T_{m+1}}=i_{m+1}$: that event can only occur if $T_{m+1}<\infty$, so, yes, you could say that condition is assumed. Last question: Norris ((1.6) on pg. 23) seems to define $h_i^j$ differently, as the probability of the event: start from state $i$ and go to state $j\in J$ while avoiding all other states in $J$, which is the same event as $X_{T_1}=i_{m+1}$, hence the last equality is valid. I'll answer the first question separately. – Mick A Dec 15 '15 at 02:13
  • @Anoldmaninthesea. Most references on the Strong MP give the (deterministic) stmt as in Norris and sometimes add that basically the SMP means you can assume the process restarts from step $0$ when the conditional event occurs. I can't find a proof of that more generalised stmt. To show this type of non-deterministic version you could do something like $$P(X_{T_{m+1}}=b|X_{T_m}=a)=\sum_{k=1}^\infty P(T_{m+1}=T_m+k,X_{T_{m+1}}=b|X_{T_m}=a))=\sum_{k=1}^\infty P(T_{m+1}=T_m+k,X_k=b|X_{T_m}=a)$$. Now $T_{m+1}=T_m+k,X_k=b$ is a fn $g$ say of $X_{T_m+1},...,X_{T_m+k}$. We can ... – Mick A Dec 15 '15 at 02:44
  • @Anoldmaninthesea. ... add an inner sum over all combinations $z_1,...,z_k$ such that $g(z_1,...,z_k)$ occurs. This turns our probability into the more deterministic form $P(X_{T_m+1}=z_1,...,X_{T_m+k}=z_k|X_{T_m}=a)$ which can then be manipulated via the chain rule (eg. $P(A,B,C|D)=P(A|B,C,D)P(B|C,D)P(C|D)P(D)$) and apply the regular Markov property to get the form required by the SMP. It's a bit messy as you can see - though there might be better ways to prove it. But the principle holds with SMP that it's just like restarting the process. ... – Mick A Dec 15 '15 at 02:45
  • @Anoldmaninthesea. ... Or another way to think is to say the SMP is just like the regular MP but with a random conditional event, which must be a "stopping time". – Mick A Dec 15 '15 at 02:46
  • @Mick, for those summands where $k>1,$ could we write $${T_{m+1}=T_m+k,X_{T_{m+1}}=b}={(X_{T_m+1},\ldots,X_{T_m+k-1},X_{T_m+k})\in (J^{c})^{k-1}\times({b}\cap J)}$$ in order to get

    $\begin{aligned}&{\color{white}=}\Bbb P(T_{m+1}=T_m+k,X_{T_{m+1}}=b\mid X_{T_m}=a)\&\overset{\text{strong MP}}{=}\Bbb P(T_1=T_0+k,X_{T_1}=b\mid X_0=a)\&=[T_0=0]\&=\Bbb P(T_1=T_0+k,X_{T_1}=b\mid X_{T_0}=a)\end{aligned}$

    and just take the sum over all $k\ge 1$ to immediately obtain the claimed result? (In my script, at least, $T_0$ is defined to be $0$.)

     – PinkyWay Feb 16 '24 at 15:44