0

In general, what will happen if the Markov chain does not start with an equilibrium probability mass vector? Feel curious about it. In many theorems, we always assume that the Markov chain starts with an equilibrium probability vector. If we relax this prerequisite, will those conclusions still hold?

For example, suppose X$_n$ is an irreducible non-null persistent Markov chain with transition matrix $\textbf{P}$ and stationary distribution $\pi$. Suppose further that X$_n$ has distribution $\pi$ for every n. And define the "reversed chain" Y by Y$_n$ = X$_{N-n}$ for 0 $\leq$ n $\leq$ N.
There is a theorem said that " The sequence Y is a Markov chain with
$\mathbb{P}$(Y$_{n+1}$ = j | Y$_n$ = i) = ($\pi_j$/$\pi_i$)p$_{ji}$.

Can we still say Y is a Markov chain if Markov chain X does not start with the stationary distribution $\pi$?

Jonny
  • 433
  • 2
  • 9

1 Answers1

1

Yes, it's still a Markov chain, but it's no longer time-homogeneous: $$\ \mathbb{P}\left(Y_{n+1}=i_{n+1}\left\vert Y_n=i_n, Y_{n-1}=i_{n-1},\dots,Y_0=i_0\right.\right)\\ = \left(\frac{\pi_{i_n}^\left(N-n\right)}{\pi_{i_{n+1}}^\left(N-n-1\right)}\right)p_{i_ni_{n+1}}\ ,$$ where $ \pi^\left(r\right)=P^r\pi^\left(0\right)\ $. When $\ \pi^\left(0\right)=\pi\ $, the stationary distribution, then $\ \pi^\left(r\right)=\pi\ $, for all $\ r\ $, and $\ Y\ $ is the time-homogeneous Markov chain you've already mentioned. If $\ X\ $ is aperiodic (as well as satisfying all the other conditions you mention), then for sufficiently large $\ r\ $, $\ \pi^\left(r\right)\ $ will be a good approximation to $ \pi\ $, and $\ Y\ $ will initially be close to time-homogeneous, with transition probabilities close to those of the stationary case. As time increases, however, depending on how far $\ \pi^\left(0\right)\ $ departs from stationarity, it will depart further and further from homogeneity.

lonza leggiera
  • 28,646
  • 2
  • 12
  • 33