0

This is a property of Poisson process. But I will ask about it for a more general process.

For a stochastic process $X$ with continuous time and discrete state space, if $\forall i$ in the state space $$ P(X_{t+h}=i | X_t=i) = 1 - r_{ii}(t) h + o(h) $$ and for $\forall j \neq i$, $$ P(X_{t+h}=j | X_t=i) = r_{ij}(t) h + o(h). $$

Is the above conditions equivalent to the time staying at state $i$ having an exponential distribution with parameter $r_{ii} h$?

Is the process necessarily Markovian?

Thanks!

Tim
  • 47,382

1 Answers1

1

No. In both cases the problem is similar, which is that the transition densities might depend on some other information than the value of $X_t$.

For a simple example, call $\mathfrak F_t$ the sigma-algebra of the events before time $t$, and $Y_t$ the last state visited before $X_t$. More precisely, let $Y_t=X_{D_t^-}$, where $D_t=\sup\{s\leqslant t\mid X_{s}\ne X_t\}$. Assume that $$ \mathbb P(X_{t+h}=i\mid X_t=i,Y_t=k,\mathfrak F_t)=1-r_{ii}^k(t)h+o(h). $$ Then the assertion in the question hold, where $r_{ii}(t)$ is some linear combination of the coefficients $(r_{ii}^k(t))_k$ but, as soon as $r_{ii}^k(t)$ does depend on $k$, $(X_t)_t$ is not Markov anymore.

The question about exponentially distributed sojourn times can only make sense when $r_{ii}(t)$ and $r_{ij}(t)$ do not depend on $t$. Even in this case, in the example given above, the sojourn time at $i$, conditionally on the previous state visited being $k$ is exponentially distributed with parameter $r_{ii}^k$, but the sojourn time at $i$ is not exponentially distributed.

Did
  • 279,727
  • Thank you. How can we show that the sojourn time at $i$ conditionally on $Y_t=k$ have exponential distribution with parameter $r^k_{ii}$? – Tim Feb 07 '13 at 13:40
  • Are you asking why a nonnegative random variable $T$ such that, for every $t\geqslant0$, $P(T\geqslant t+h\mid T\geqslant h)=1-rh+o(h)$ is exponentially distributed with parameter $r$? – Did Feb 07 '13 at 13:50
  • Yes. Looks similar to memoryless property but not exactly the same? Also why can't the question about exponentially distributed sojourn times make sense when $r_{ii}(t)$ and $r_{ij}(t)$ depend on t? – Tim Feb 07 '13 at 14:27
  • Because for every nonnegative random variable $U$ with continuous density, one has $P(U\geqslant t+h\mid U\geqslant t)=r(t)h+o(h)$ for some function $r:t\mapsto r(t)$ hence the only way to limit oneself to exponential distributions is to assume $r(t)$ is constant. – Did Feb 07 '13 at 14:45
  • If $r(t)$ is piecewise constant for $t \leq t_0$ and $t > t_0$ respectively, does $U(t)$ have two different exponential distributions for $t \leq t_0$ and $t > t_0$ respectively? – Tim Mar 03 '13 at 14:46
  • The description $P(U\geqslant t+h\mid U\geqslant t)=1-r(t)h+o(h)$ for every $t\geqslant0$ (sorry about the typo $r(t)h$ instead of $1-r(t)h$ in my last comment) is equivalent to $P(U\geqslant t)=\exp\left(-\int\limits_0^tr(s)\mathrm ds\right)$ for every $t\geqslant0$. – Did Mar 03 '13 at 16:01