1

Consider a continuous-time Markov chain (CTMC) $X$ on a countably infinite state space $S$. The CTMC is irreducible and all the states are positive recurrent. Let $T(i,j)$ be the first passage time to state $j$ given that it starts in state $i$. $T(i,i)$ should be understood to be the return time to state $i$.

Since all states are positive recurrent, we have for all $i \in S$,

\begin{equation} \mathbb{E}[T(i,i)] < \infty \quad \Leftrightarrow \quad \mathbb{P}(T(i,i) < \infty) = 1. \end{equation}

Intuitively, it seems that we should also have for all $i,j \in S$,

\begin{equation} \mathbb{E}[T(i,j)] < \infty \quad \Leftrightarrow \quad \mathbb{P}(T(i,j) < \infty) = 1. \end{equation}

However, I don't see an immediate way to prove this. Possibly by using that the $\mathbb{P}(X(t) = j \mid X(0) = i) > 0$ for all $i,j \in S$ and $t > 0$ by positive recurrence?

Ritz
  • 1,693
  • 9
  • 18
  • 1
    Maybe my solution here will help: http://math.stackexchange.com/questions/313124/markov-chain-with-finite-positive-recurrent-states/313173#313173 –  Feb 20 '16 at 16:13
  • I'm not sure what you're asking. By definition, if the process is irreducible and positive recurrent then $\mathbb E[T(i,j)]$ is finite for all $i,j$. – Math1000 Feb 29 '16 at 23:51
  • @Math1000 Usually positive recurrence is defined by stating that state $i$ is positive recurrent if $\mathbb{E}[T(i,i)] < \infty$. How would you define the positive recurrence of state $i$ in terms of $\mathbb{E}[T(i,j)] < \infty$ (so in terms of $j$)? Can you point me to a reference that defines positive recurrence in the way you described? – Ritz Mar 01 '16 at 08:45
  • Not by "definition," perhaps, but the argument follows readily from every state being positive recurrent and thus being visited infinitely often with probability one. – Math1000 Mar 01 '16 at 09:18
  • @Math1000 Being visited infinitely often with probability one establishes recurrence, not positive recurrence. Positive recurrence requires the expectation of the return times to be finite. Nonetheless, I was able to prove it using the answer provided by Byron Schmuland for the discrete-time case. – Ritz Mar 01 '16 at 09:33

0 Answers0