Hitting Time Markov Chain

Question

Let$\left\{{X_{n}: n\in \mathbf{N}}\right\}$ be a Markov Chain in discrete time, with the hitting time being defined as $\displaystyle H^A=\inf\left\{{n\geq 0 : X_{n}\in A}\right\}$. Assuming $i\not \in A$, how do I prove that $\mathbf{E}\left(H^A|X_1=j;X_0=i\right)= 1 + \mathbf{E}\left(H^A|X_0=j\right) $ ?

Intuitively I understand why it is so.

I seem to have found a proof but it relies on :
$\displaystyle \sum^{\infty}_{n=0} {\Pr \left( H^A=n-1|X_0=j\right)=1}$, when it is

$\displaystyle \sum^{\infty}_{n=0} \left(\Pr\left(H^A=n-1|X_0=j\right)\right)+\Pr\left(H^A=\infty|X_0=j\right)=1$

Is this last equality the correct one, or am I wrong? — An old man in the sea., Nov 13 '13 at 22:07
Not anymore. I found out why a long time ago. But thanks anyway ;) — An old man in the sea., Jan 16 '14 at 22:16
@Lost1 Would you like to help me on this other one? http://mathoverflow.net/questions/154534/log-likelihood-in-regime-switching — An old man in the sea., Jan 16 '14 at 22:21
me neither. But to answer my question you do not need to be one , I think. The formula is just in the first pages of the paper and stated as if it's sth trivial to derive... (but yes... you might be right) — An old man in the sea., Jan 16 '14 at 22:36

score 0 · Answer 1 · answered Jan 16 '14 at 11:13

I think I understand your confusion

$E(H^A|X_1=i,X_j=0) = E(H^A1_{\{H_A\geq 1\}}|X_1=j) = E(1+\tilde{H}^A|X_1=j)= 1+E(H^A|X_1=j)$

where $\tilde{H}^A$ is an independent copy of $H^A$.

The confusion comes from the notation. I do not like the notation you use. I think things are much clearer if you use the notaiton

$E_i$ to mean $E(.. |X_0=i)$. The difference is the following. In the latter, everything live on 1 probability space and $E_i$ for each $i$ in the sample space is a family of probability spaces. Markov property written in this form

$E_i(H^A\mathbb{1}_{\{H^A\geq 1\}}|X_1=j) = E_j(H^A\mathbb{1}_{\{H^A\geq 1\}}\circ\theta_1)=E_j((1+H^A\mathbb){1}_{\{H^A\geq 0\}})$

where $\theta_1$ is shifting time by 1. Hitting time under the operator behaves like $H^A\mathbb{1}_{\{H^A\geq 1\}}\circ\theta_1 =(1+H^A\mathbb){1}_{\{H^A\geq 0\}}$

Hitting Time Markov Chain

1 Answers1