1

Given a Markov chain $\{X_n \mid n \in \{0, 1, \ldots\}\}$ with states $\{0, \ldots, N\}$, define the limiting distribution as $$ \pi = (\pi_0, \ldots, \pi_N) $$ where $$ \pi_j = \lim_{n \to +\infty} \mathbb{P}\{X_n = j \mid X_0 = i\} $$

I am confused as to why we condition on $X_0 = i$. What kind of a role does the initial state play? My textbook offers no explanation.

d125q
  • 2,370
  • 17
  • 19
  • 1
    If the chain is finite and irreducible, then the initial state does not matter for the definition (the limit is the same for all initial states). On the other hand, if the chain is reducible, you could have multiple limit distributions, which would arise differently depending on which component you start in. – Alex R. Mar 17 '15 at 19:46
  • 1
    Consider a chain on a two element state space ${a,b}$ that has transition probabilities of zero between different states. The limiting distribution very much depends on where the chain starts! – sourisse Mar 17 '15 at 21:03
  • 1
    How would you define $\pi_j$ otherwise? – Did Mar 17 '15 at 21:04
  • @Did: I thought of defining it without the condition $X_0 = i$. As others have pointed out, this wouldn't make much sense. – d125q Mar 17 '15 at 21:13
  • As it happens, under some conditions, starting from any distribution $\nu$ and defining $\pi$ as the limit of the distribution of $X_n$ conditionally on $X_0(P)=\nu$ yields the same $\pi$... But you did not say the definition of $\pi$ without conditioning on $X_0=i$ you had in mind? – Did Mar 17 '15 at 21:16
  • @Did: The definition I had in mind would've been simply the probability of finding the process in state $j$ after it has run for long enough time, i.e. $\lim_{n \to +\infty} \mathbb{P}{X_n = j}$. – d125q Mar 17 '15 at 21:28
  • But $P(X_n=j)$ needs an initial distribution to be simply defined. – Did Mar 17 '15 at 21:32

1 Answers1

1

Consider the following system, with $N=3$ :

$$M= \begin{bmatrix} 0.5 & 0.5 & 0 & 0 \\ 0.4 & 0.6 & 0 & 0 \\ 0 & 0 & 0.3 & 0.7 \\ 0 & 0 & 0.2 & 0.8 \\ \end{bmatrix}$$ where $$M_{i, j} = \mathbb{P}\left\{X_n = j \mid X_{n-1} = i\right\}$$

Notice that $$M^\infty = \begin{bmatrix} \frac 49 & \frac 59 & 0 & 0 \\ \frac 49 & \frac 59 & 0 & 0 \\ 0 & 0 & \frac 29 & \frac 79 \\ 0 & 0 & \frac 29 & \frac 79 \\ \end{bmatrix}$$

Can you say that $M^\infty\,_{i,j}$ doesn't depend on $i$?

DanielV
  • 23,556
  • Thank you for clearing this up to me with an example. If, however, the chain is regular, then the limiting distribution is unique, correct? – d125q Mar 17 '15 at 21:15
  • @d125q I believe that is correct. A regular MC is one where all entries are positive (nonzero) so every state can transition to every other state so the initial condition eventually won't matter. – DanielV Mar 17 '15 at 22:01
  • "A regular MC is one where all entries are positive" No. – Did Mar 18 '15 at 06:24
  • @Did Thanks, you are right, I misread : https://www.math.ucdavis.edu/~daddel/linear_algebra_appl/Applications/MarkovChain/MarkovChain_9_18/node2.html it should be $\exists n ~ A^n$ has no zero elements, which still leads to the fact that every state can (eventually) transition to every other state. Although with a nondiscrete time markov chain, I think there could be a difference (two states might take a rational and an irrational time to transition to each other , so no integer would have $A^n$ being positive). – DanielV Mar 18 '15 at 12:51