2

In the Information Bottleneck (IB) paper (https://arxiv.org/pdf/physics/0004057.pdf). Using lagrange multipliers we need to solve $\frac{\delta F}{\delta p(\tilde{x}|x)}=0$, where $F = I(X;\tilde{X})+\beta d(x,\tilde{x})$. By substituting $I(X;\tilde{X})=\sum_{x,\tilde{x}}p(x,\tilde{x})log(\frac{p(\tilde{x}|x)}{p(\tilde{x})})$ and taking the derivative of $F$, the paper writes down the solution as follows, $\frac{\delta F}{\delta p(\tilde{x}|x)}= p(x)[log(\frac{p(\tilde{x}|x)}{p(\tilde{x})})+1-\frac{1}{p(\tilde{x})}\sum_{x'}p(x')p(\tilde{x}|x')+\beta d(x,\tilde{x})+\frac{\lambda(x)}{p(x)}]$.

Question: I don't understand how it gets the term "$-\frac{1}{p(\tilde{x})}\sum_{x'}p(x')p(\tilde{x}|x')$"? The rest of the terms are clear but I cannot understand only this term.

Mah
  • 21

0 Answers0