2

If I have a Markov chain with finite positive recurrent states $\in S$, then that means starting from a given state $y$, the expected number of steps to return to state $y$ is finite.

Now, if I start at state $j\ne y$ where $y,j\in S$, then I am under the impression that the expected number of steps to reach state $y$ given I start in state $j$ is finite.

I know that state $j$ is positive recurrent too, but I am not sure how to prove the previous statement mathematically?

Solver
  • 173
  • I am not sure I know what to expect to return to state y in finitely many steps really means. Can you explain? – Did Feb 24 '13 at 19:36
  • It means that if I am at state y, then $E_y (T_y)\lt \infty$ where $T_y=min(n\gt 0:X_n=y)$ is the hitting time for state y. I hope that helped. – Solver Feb 24 '13 at 19:52
  • You might want to modify accordingly your question. – Did Feb 24 '13 at 19:56
  • They mean the same thing. In the question I am explaining $E_y(T_y)<∞$ in words. – Solver Feb 24 '13 at 20:34
  • No they do not. Actually I might never have seen the phrase to expect to return to some state in finitely many steps to mean that the expectation of the return time is finite. Note that if one does return to some state, this is always in finitely many steps. – Did Feb 24 '13 at 20:55
  • Are you familiar with the term "null recurrent"? If a state is null recurrent, then the expected return time is infinite. – Solver Feb 24 '13 at 21:25
  • let us continue this discussion in chat – Solver Feb 24 '13 at 21:39
  • No. (Anyway your "Are you familiar" comment is a strong incentive not to.) – Did Feb 24 '13 at 21:46
  • I am not sure what you mean by the last comment, but I am sorry if you felt offended. I didn't mean to offend you, I was just wondering whether you read about "null recurrence". For me, I wasn't familiar with this term until recently.

    BTW, the "let us continue this discussion in chat" sentence was generated automatically when I moved the discussion to the chat section. Sorry if that annoyed you.

     – Solver Feb 24 '13 at 22:08
  • Please feel free to comment if you like, and sorry again for any miscommunication. – Solver Feb 24 '13 at 22:10
  • Rather than feeling sorry for this and that and mentioning obvious facts about the "chat" comment, it would be more useful to correct your post. – Did Feb 24 '13 at 22:12
  • I realized that what you said about the two statements being different is correct. I corrected the post above. Thanks for the clarification. – Solver Feb 24 '13 at 22:34
  • Great. Good job. I upvoted the question (and the answer). – Did Feb 25 '13 at 07:25

1 Answers1

3

From the strong Markov property at $T_j$, and the fact that $y$ is positive recurrent we have $$\infty>\mathbb{E}_y(T_y)\geq \mathbb{E}_y(T_y\,1_{(T_j<T_y)}) = \mathbb{E}_y([T_y\circ\theta_{T_j}+T_j]\,1_{(T_j<T_y)})$$ $$\geq \mathbb{E}_y([T_y\circ\theta_{T_j}]\,1_{(T_j<T_y)})=\mathbb{P}_y(T_j<T_y)\,\mathbb{E}_j(T_y). $$ If it is possible to go from state $y$ to state $j$, then $\mathbb{P}_y(T_j<T_y)>0$ and we conclude that $\mathbb{E}_j(T_y)<\infty.$

You must assume that the states communicate. For example, if all states are absorbing then they are all positive recurrent, but $\mathbb{E}_j(T_y)=\infty$ for $j\neq y$.

  • For this part: $ℙy(T_j\lt T_y)_j(T_y)=_y(T_y1{(T_j\lt T_y)})\lt \infty$.

    Shouldn't it be: $ℙy(T_j\lt T_y)_j(T_y)\le _y(T_y1{(T_j\lt T_y)})\lt \infty$?

     – Solver Feb 24 '13 at 20:31
  • And thanks for the clarification. – Solver Feb 24 '13 at 21:25
  • @Solver You are right, that should be an inequality. Apologies! –  Feb 25 '13 at 18:28
  • This line of reasoning works for a countably infinite state space, correct? I do not see why $\mathbb{E}y[T_y] \ge \mathbb{E}_y[T_y 1{T_j < T_y}]$. Say $\mathbb{E}y[T_y] = 10$, and state $j$ is at least 20 steps away from state $y$, then is $\mathbb{E}_y[T_y 1{T_j < T_y}] \ge 20$? Can you explain the notation $[ T_y \circ \theta_{T_j} + T_j ]$ please? Thank you! – Ritz Feb 21 '16 at 12:08
  • The random variables satisfy the pointwise inequality $T_y\geq T_y,1_{{T_j<T_y}}$, since the indicator function only takes the values 0 and 1. –  Feb 21 '16 at 15:46
  • 1
    @Ritz $\theta$ is the shift operator, so $T_j+T_y\circ\theta_{T_j}=\inf(t≥T_j:X_t=y)$, the first time after hitting state $j$ that the process hits state $y$. –  Feb 21 '16 at 16:22