2

Suppose I have a two-state continuous-time Markov chain $M$ with rate matrix $Q$.

$$ Q = \begin{bmatrix} -q_{01} & q_{01} \\ q_{10} & -q_{10} \end{bmatrix} $$

Now consider a Poisson process $P$ whose rate is $\lambda_0$ when $M$ is in state 0, and $\lambda_1$ when $M$ is in state 1. Then $P$ is a Markov-modulated Poisson process. I want to calculate the probability that no events have occurred in $P$ by time $t$.

Here's my reasoning so far. Suppose that $M$ always starts in state 0. Let $P_0(t)$ and $P_1(t)$ be the probabilities that $M$ is in state 0 and 1, respectively, at time $t$. Then the expected rate of $P$ at time $t$ is

$$ \lambda_0 P_0(t) + \lambda_1 P_1(t).$$

It follows that the probability of no events occurring before time $t$ is

$$ \exp\left[ -\int_{s=0}^t (\lambda_0 P_0(s) + \lambda_1 P_1(s)) \, ds\right]. $$

However I am uncertain if it makes sense to use the expected rate this way, to get at the probability of no events occurring. I would like to know if this makes sense, and if not, where did I go wrong?

rmccloskey
  • 153
  • 6

1 Answers1

1

I found the result I needed in "Fischer, Wolfgang, and Kathleen Meier-Hellstern. "The Markov-modulated Poisson process (MMPP) cookbook." Performance evaluation 18.2 (1993): 149-171.".

Let $P_{ij}(n, t)$ be the probability that, at time $t$, $n$ arrivals have occurred and the chain is in state $j$, given that the chain began in state $i$ at time 0. Then the matrix $P(n, t)$ satisfies the differential equation

$$ P'(n, t) = P(n, t) (Q - \Lambda) + P(n-1, t) \Lambda, \quad P(0, 0) = I,$$

where $Q$ is the transition matrix of the Markov chain and $\Lambda = \text{diag}(\lambda_1, \ldots, \lambda_n)$ is the diagonal matrix of the arrival rates. In particular, the row sums of $P(0, t)$ are the probabilities that no arrivals have occurred by time $t$.

rmccloskey
  • 153
  • 6