1

I had a simple question yesterday when I was trying to solve an exercise on a reducible,aperiodic Markov Chain. The state spase S was $$S=\{1,...,7\}$$ and we could partition it into two closed classes $\{5,6,7\} \bigcup \{3,4\}$ but the class $\{1,2\}$ was open (thus transient). The exercise was giving the transition matrix and was asking to find the stationary distributions. At the end it was also asking to find the limit distribution, given that the INITIAL distribution was $$P[X_0=1]=P[X_0=2]=\frac{1}{2}$$ I only know what to do in case the initial probability gives mass 1 to some state (I find the absorption probabilities in each closed class, etc..) but when I have initial distribution which gives mass to both of these transient states I do not know what to do. Any helpful ideas/theorems about this?? Thanks a lot !

$$ \begin{pmatrix} 1/3 & 1/6 & 1/3 & 0 & 0 & 1/6 \\ 3/5 & 0 & 0 & 1/5 & 1/5 & 0 & 0 \\ 0 & 0 & 1/2 & 1/2 & 0 & 0 & 0 \\ 0 & 0 & 1/4 & 3/4 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 0 & 0 & 1/4 & 3/4 \\ 0 & 0 & 0 & 0 & 1/2 & 0 & 1/2 \end{pmatrix} $$

Petros Karajan
  • 374
  • Giving the transition matrix would make things easier (to answer) here. An answer of the kind "take 1/2 of the limit distribution for the case of giving full probability to the state 5 and also take 1/2 of the limit distribution for the case of giving full probability to the state 6 and add them" is not really beautiful. – dan_fulea Jan 18 '19 at 18:03
  • OK,let's suppose we have the following example in page 1 : https://www.win.tue.nl/~resing/SOR/handout.pdf and you want to answer question 2) . – Petros Karajan Jan 18 '19 at 18:20
  • It may be useful to use mathjax, and include the example in the original post, https://math.meta.stackexchange.com/questions/5020/mathjax-basic-tutorial-and-quick-reference – dan_fulea Jan 18 '19 at 18:21
  • I will write it afterwards. Do you have any idea about the solution? – Petros Karajan Jan 18 '19 at 18:26
  • It should be a "linear play". More exactly, in some notation i am used to, the initial distribution is a row vector, shape $N\times 1$, where $N$ (six or seven in our case) is the number of states, and $A$, the transition matrix, is a an $N\times N$ matrix. So we need to compute something like $\lim_n qA^n$ for $n\to \infty$. This expression is linear in $q$. So if we know the answer for the canonical base $e_1=[1,0,0,0,\dots]$, $e_2=[0,1,0,0,\dots]$, ... then we need to linearly combine them using the weights in $q$. – dan_fulea Jan 18 '19 at 18:33
  • @dan_fulea !!! Thank you a lot!! Yes,your hint was right, indeed, if we know it about the case of dirac masses on each j, then a linear combination gives the general case. I will right the answer on LaTex code now. – Petros Karajan Jan 18 '19 at 18:47

1 Answers1

2

As dan_fulea noticed above, it suffices to calculate the case when $ P[X_0=1]=1 $ and $ P[X_0=2]=1 $ . The first is already treated in the notes: it gives the limit vector $$ \begin{pmatrix} 0, & 0, & \frac{11}{51}, & \frac{22}{51}, & \frac{18}{221}, & \frac{24}{221}, & \frac{36}{221} \end{pmatrix} $$ For the case $ P[X_0=2]=1 $ , since $q_{2,E_1}=\frac{10}{17}$ and $ q_{2,E_2}=\frac{7}{17}$, we get the limit vector : $$\begin{pmatrix} 0, & 0, & \frac{10}{51}, & \frac{20}{51}, & \frac{21}{221}, & \frac{28}{221}, & \frac{42}{221} \end{pmatrix} $$ Hence, the final answer to our question is : $$ \frac{1}{2}\begin{pmatrix} 0, & 0, & \frac{11}{51}, & \frac{22}{51}, & \frac{18}{221}, & \frac{24}{221}, & \frac{36}{221} \end{pmatrix} + \frac{1}{2} \begin{pmatrix} 0, & 0, & \frac{10}{51}, & \frac{20}{51}, & \frac{21}{221}, & \frac{28}{221}, & \frac{42}{221} \end{pmatrix} $$

Petros Karajan
  • 374