Why is the stationary distribution of a Markov Chain one over the mean sojourn time?

Question

I am reading the book "Markov Chains and Mixing Times" by Levin et al. and i have a problem with proposition 1.14, page 12.

There a stationary distribution is constructed by defining for fixed initial state z

$$\tilde \pi(y):=\mathbf E_y(\text{number of visits to y before returning to z})=\sum_{t=0}\mathbf P(X_t=y,\tau_z^+>t)$$

$\tau_z^+$ is the first return time to z.

The proof is all well, but in the end a stationary distribution $\pi$ is constructedby normalizing such that $$\pi(x)=\frac{\tilde \pi(x)}{\mathbf E_z(\tau_z^+)}$$

Then, without further comment it is stated that this means in particular that $$\pi(x)=\frac{1}{\mathbf E_x(\tau_x^+)}$$

How do i see this?

Answer 1

The statement is true, but I think the way it is presented is a mistake by the authors. What is obvious is that

$$\pi(z) = \frac{1}{\mathbb{E}_z (\tau_z^+)}.$$

Indeed, starting from $z$, the Markov chain visits $z$ exactly once (at time $t=0$) before returning to $z$, so $\tilde{\pi} (z) = 1$.

However, the stationary measure we got depends a priori on the state $z$ chosen to construct it. If we use another state $z'$, then we get another stationary measure $\pi'$ such that $\pi'(z') = \frac{1}{\mathbb{E}_{z'} (\tau_{z'}^+)}$, but there is no guarantee that $\pi'(z') = \pi(z')$! This is actually false for certain examples of Markov chains (take e.g. a Markov chain on two states with the identity as the transition matrix).

What saves us is that, if the Markov chain is irreducible, there is indeed a unique stationary probability measure, which implies that $\pi = \pi'$. Anyway, at this point, the proof must somehow use that fact that the Markov chain is irreducible.

I think that this is a mistake by the authors since they do not explicitely mention the irreducibility at this point of the proof, and the uniqueness is only proved in the following section.