2

I am not able to understand the reasoning behind the load-balance equation of the birth-death processes (Markov chain) . The equation is $\pi_ib_i = \pi_{i+1}d_{i+1}$ , where the symbol have their usual meanings ie ( $b_i$ is the probability of going from state $i$ to $i+1$ and $d_{i+1}$ is the probability of coming from state $i+1$ to state $i$ ) .

What the book says :

For a birth-death process, the balance equations can be substantially simplified. Let us focus on two neighboring states, say, $i$ and $i+1$. In any trajectory of the Markov chain, a transition from $i$ to $i+1$ has to be followed by a transition from $i + 1$ to $i$, before another transition from $i$ to $i + 1$ can occur. Therefore, the frequency of transitions from $i$ to $i + 1$, which is $π_ib_i$, must be equal to the frequency of transitions from $i + 1$ to $i$, which is $π_{i+1}d_{i+1}$. This leads to the local balance equations.

If I simply go on and write a load balance equation at node $i$ the equation must be $$\pi_i = \pi_{i-1}b_{i-1} + \pi_{i+1}d_{i+1}+\pi_i(1-b_{i-1}-d_{i+1})$$ but if I go for solving these I can never reach the earlier the simple equation given earlier.

I don't understand why the frequencies should be same . Suppose I have more probability of going from $i$ to $i+1$ and less of returning then how can they be balanced in the long run . I know my reasoning is wrong but I am not understanding the fallacy . Please help me here .

PS: I am studying markov chains for the first time

Thanks in advance

Did
  • 279,727
abkds
  • 2,210

1 Answers1

4

The book suggests to look at what goes through an edge. Indeed, at equilibrium, since the only way to go from $\{j\mid j\leqslant i\}$ to $\{j\mid j\geqslant i+1\}$ or from $\{j\mid j\geqslant i+1\}$ to $\{j\mid j\leqslant i\}$ is to use the edge $\{i,i+1\}$, the fluxes through this edge in the direction $i\to i+1$ and in the direction $i+1\to i$ must compensate, that is, $$ \pi_ib_i = \pi_{i+1}d_{i+1}.\tag{$\ast$} $$ Your idea, on the other hand, is to look at what leaves, what arrives and what stays at a node. Unfortunately, there is a mistake in your computations, and this approach leads actually to $$ \pi_i = \pi_{i-1}b_{i-1} + \pi_{i+1}d_{i+1}+\pi_i(1-b_{i}-d_{i}), $$ the last term on the RHS counting what stays at node $i$. Usually though, one equates what leaves and what arrives at a node, in which case one gets the equivalent identity $$ \pi_{i-1}b_{i-1} + \pi_{i+1}d_{i+1}=\pi_i(b_{i}+d_{i}).\tag{$\dagger$} $$ And now the Grand Finale: the systems $(\ast)$ and $(\dagger)$ are in fact equivalent... hence each yields the correct stationary distribution $$ \pi_i=\pi_0\cdot\prod_{j=1}^i\left(\frac{b_{j-1}}{d_j}\right), $$ for the unique positive factor $\pi_0$ which makes that the mass of $(\pi_i)$ is $1$.

Did
  • 279,727