1

  1. In a continuous-time Markov chain, I was wondering why the holding time and the next state are independent? Are the independence a conditional one given the current state?

  2. Quoted from Ross's Stochastic processes:

    The amount of time the process spends in state $i$, and the next state visited, must be independent random variables. For if the next state visited were dependent on $\tau_i$, then information as to how long the process has already been in state $i$ would be relevant to the prediction of the next state—and this would contradict the Markovian assumption.

    One can also find identical claim at another book here with more
    context available.

    I don't understand why if the two are dependent, the Markov property is violated.

Thanks and regards!

Srivatsan
  • 26,311
Tim
  • 47,382

2 Answers2

2

The next state depends on the recent states, not how long you stayed in them. Notice that this is implicit in the state transition matrix.

Emre
  • 2,783
  • Thanks! Could you elaborate on "this is implicit in the state transition matrix"? – Tim May 02 '11 at 00:19
  • There is no information in the state transition matrix about holding times. It merely tells you the distribution of the next state given the past states. – Emre May 02 '11 at 00:21
2

For question 1, the answer is yes. You might first consider the case of 2 possible "next states", which corresponds to showing that if $U,V$ are two independent exponential random variables (with possibly different rates), then $\min(U,V)$ is independent of the event $U < V$. Counterintuitive but true.

For question 2, consider a simple example: suppose $X_t$ is a process that, whenever it enters a certain state $a$, it does one of two things: waits 5 seconds and then transitions to $b$, or waits 10 seconds and then transitions to $c$. Here the holding time is completely correlated with the next state. Such a process cannot be Markov. For instance, we have $P(X_{20} = b | X_{17} = a) > 0$, but $P(X_{20} = b | X_{17} = a, X_{14} \ne a) = 0$.

Essentially, information about the holding time is information about the history of the process (where it was at an earlier time), and in a Markov process this is not allowed to prejudice where it goes next.

Nate Eldredge
  • 97,710
  • Thanks! (1) I was wondering why given the current state, it corresponds to independent U and V? (2) When U and V are independent exponential r.v.s, why min(U,V) is independent of the event U<V? (3) In the example, at state a, the intervals in the two possibilities are 5 and 10 seconds, while in $P(X_{20} = b | X_{17} = a) $ and $P(X_{20} = b | X_{17} = a)$, the intervals are 3. How do you know the $P(X_{20} = b | X_{17} = a) > 0$ and $P(X_{20} = b | X_{17} = a) > 0$? – Tim May 09 '11 at 05:24