0

Suppose I have an irreducible and recurrent Markov chain in a state space with at least 2 states, given by a transition matrix $ (P)_{ij} $. I would like to show that the derived transition matrix

$$ \hat{P_{ij}} = \begin{cases} 0 & i = j \\ (1 -P_{ii})^{-1}P_{ij} & i \neq j\end{cases}$$

gives a Markov chain which is also irreducible.

My current reasoning is as follows. Firstly, this transformation makes sense as $ P_{ii} \neq 1, \forall i \in S$, otherwise the chain clearly isn't irreducible.

I've reduced to two cases. If $i = j$, then $\hat{P_{ii}}(0) = 1 > 0$. But I'm stuck in the case if $ i \neq j$. My idea is that because $P$ is already irreducible, since the new matrix is essentially the old one for probabilities between distinct states (bar the non-zero constant $(1 - P_{ii})^{-1}$), then we can still take a path between two states without $\hat{P_{ij}(n)}$ reducing to zero for some $n > 0$. Is there any way to show this more explicitly? I can't really find a way to put Chapman-Kolmogorov to good use either.

Afiq Hatta
  • 239

1 Answers1

1

Let $i,j$ be distinct states. Since the Markov chain is irreducible, there exist states $i_1,\ldots,i_n$ such that $$ P_{i,i_1}\left(\prod_{k=1}^{n} P_{i_k,i_{k+1}}\right)P_{i_n,j} > 0. $$ If the $i_k$ are distinct, then as $\hat P_{x,y}\geqslant P_{x,y}$ for $x\ne y$, we have $$ \hat P_{i,i_1}\left(\prod_{k=1}^{n} \hat P_{i_k,i_{k+1}}\right)\hat P_{i_n,j} \geqslant P_{i,i_1}\left(\prod_{k=1}^{n} P_{i_k,i_{k+1}}\right)P_{i_n,j} > 0. $$ If $i_k=i_{k+1}$ for some $1\leqslant k\leqslant n-1$, then the above holds with the terms $\hat P_{i_k,i_{k+1}}$ and $P_{i_k,i_{k+1}}$ omitted. It follows that $$ \hat {\mathbb P}(\sup\{m>0: X_m = j\mid 0\}) > 0. $$

Math1000
  • 36,983