1

I have to deal with a dynamical system that looks as follows, with $H$ being the initial state (parameters next to arrows denote transition probabilities between the states $H$, $L_1$ and $L_2$):

enter image description here

I must admit that I have not much clue how to analyze such a system. I would be interested in two basic questions:

(1) If we let the dynamical system run for a long while (as the number of periods $T=1,2,3,...$ goes to infinity), which fraction of the time will be spent in each of the 3 states?

(2) Conditional on having a switch from either $L_1$ or $L_2$ to $H$, which fraction of the time do we experience a switch from $L_2$ to $H$?

Any answers how to deal with that problem would be most welcome. Thanks in advance!

Martin
  • 475

2 Answers2

1

A general result says that (under technical hypotheses which are met in the present case) the proportion of time the system spends in each state $H$, $L_1$ and $L_2$, in the long run, corresponds to the stationary distribution of this state, that is, its weight with respect to the so-called stationary distribution $\pi$. Furthermore, $\pi$ can be computed by solving the so-called balance equations, which roughly assert that, at equilibrium, as there is as much mass leaving each state than arriving on this same state.

For example, at state $H$, in one step, the mass that leaves is $\varrho_H\pi(H)+(1-\varrho_H)\pi(H)$ and the mass that arrives is $\varrho_H\pi(H)+(1-\varrho_L)\pi(L_1)+(1-\varrho_L)\pi(L_2)$. The total mass $\pi(H)+\pi(L_1)+\pi(L_2)$ is $1$, hence one gets $(1-\varrho_H)\pi(H)=(1-\varrho_L)(1-\pi(H))$, from which the value of $\pi(H)$ follows.

The same approach applied at states $L_1$ and $L_2$ yields the values of $\pi(L_1)$ and $\pi(L_2)$.

Re (2), the transition probabilities from $L_1$ to $H$ and from $L_2$ to $H$ coincide hence the proportions of times when one arrives at $H$ coming from $L_1$ and when one arrives at $H$ coming from $L_2$ are in the ratio $\pi(L_1)/\pi(L_2)$.

Did
  • 279,727
  • Thanks a lot, that is exactly the answer I was hoping for! However, just for clarification: The mass that leaves at state $H$, in one step, should just be $(1-\rho_H)\pi(H)$, no? – Martin Sep 27 '13 at 13:38
  • No, this counts the mass which goes along the loop from H to H, both as leaving H and as arriving at H. The other convention (to omit this mass altogether) is quite possible to use (although, personally, I find it less systematic), it yields a different system but the same solution, naturally. – Did Sep 27 '13 at 13:45
  • I see. But then, I guess the mass that arrives at $H$ should also include $\rho_H \pi(H)$, which you didn't include in the second line. – Martin Sep 27 '13 at 14:05
  • Your guess is right (thanks about it) and I corrected the answer. – Did Sep 27 '13 at 14:15
0

I think(!) you can reason as follows:

time in H: Th

time in L1: Tl1

time in L2: Tl2

time in H is equal to the number of times a transition to H is made:

Th=Th*pH+Tl1*(1-pL)

Tl1=Th*(1-pH)+Tl1*(plx)

Tl2=tl1*(.....

etc.

And than you can solve it for Th, Tl1 and Tl2. proportion of time spend in Th=Th/(Th+Tl1+Tl2)

Again, I'm not sure this is correct

user2520938
  • 7,235