2

I need some help about a particular notion I use as a PhD student in economics (and former maths student but no matter).

For a classical homogeneous Markov chain $(X_n)_n$ with finite space states, we can define the random variable called "first passage time". Let us denote $T_{ij}=\min\{n\in \mathbb{N}^*\ /\ X_n=j, X_{n-1} \neq j,\dots,X_1 \neq j,\ X_0=i\} $, the shortest time it takes to transit from state $i$ to state $j$ and let us denote $f_{ij}^{(n)}=\mathbb{P}(T_{ij}=n)$. We can show that $\displaystyle f_{ij}^{(n+1)}= \sum_{k \neq j} p_{ik}f_{kj}^{(n)}$ with $f_{ij}^{(1)}=p_{ij}$ are traditional transition probabilities from $i$ to $j$.

My question is the following one : is this formula still valid for a non homogeneous Markov chain that is with transition probabilities $p_{ij}(n)$ depending on time ? If not, can we adapt it ? That requires to redefine $T_{ij}$ in $T_{ij}(m)$ as well and $f_{ij}^{(n)}$ in $f_{ij}^{(n)}(m)$, the condition being $X_m=i$.

Thanks for your help.

Alexique
  • 302

1 Answers1

1

Ok, I propose something and I hope somebody will show interest in my attempt of proof.

By definition, let us set $f_{ij}(n,m)= \mathbb{P}(X_{m+n}=j , X_{m+n-1} \neq j ,... , X_{m+1} \neq j \mid X_m=i)$. For $m=0$, we have $f_{ij}(n,0)=f_{ij}^{(n)}$, the classical first passage time and let us set $p_{ij}(n)=\mathbb{P}(X_{n+1}=j \mid X_n=i)$. Hence, we have $f_{ij}(1,m)=p_{ij}(m)$. Imitating the proof of the formula in the homogeneous case, we begin by using the Law of total probability : $f_{ij}(n,m)= \sum_{k \neq j} \mathbb{P}(X_{m+n}=j , X_{m+n-1} \neq j ,... , X_{m+1} =k \mid X_m=i)=\sum_{k \neq j} \mathbb{P}(X_{m+n}=j , X_{m+n-1} \neq j ,... , X_{m+2} \neq j \mid X_{m+1} =k , X_m=i)\mathbb{P}(X_{m+1} =k \mid X_m=i) =\sum_{k \neq j} \mathbb{P}(X_{m+n}=j , X_{m+n-1} \neq j ,... , X_{m+2} \neq j \mid X_{m+1} =k) p_{ik}(m) =\sum_{k \neq j} f_{kj}(n-1,m+1)p_{ik}(m)$.

In a naive way, if I set matrices $P(m) = (p_{ij}(m))_{ij}, F(n,m)=(f_{ij}(n,m))_{ij}$, we get $F(n,m)$ as a function of $F(n-1,m+1)$ et $P(m)$. From one to the next, we should get $F(n,m)$ as a function of the $P(i)$ known and of $F(1,m+n-1)=P(m+n-1)$. So the result seems to hold, doesn'it ?

Thanks for the feedback.

Alexique
  • 302