I read that the Kullback-Leibler divergence defined as $$D_{KL} = \int_{-\infty}^{\infty}p\left(x\right)\cdot log\left(\frac{p(x)}{q(x)}\right)dx$$ can be seen as a special case of the f-divergence $$D_{f} = \int_{-\infty}^{\infty}p\left(x\right)\cdot f\left(\frac{p(x)}{q(x)}\right)dx$$ if you use $f = x \cdot log(x)$. But I don't see why because when I plug it in, then $$D_{f} = \int_{-\infty}^{\infty}p\left(x\right)\cdot f\left(\frac{p(x)}{q(x)}\right)dx = \int_{-\infty}^{\infty}p\left(x\right)\cdot \frac{p(x)}{q(x)} \cdot log\left(\frac{p(x)}{q(x)}\right)dx \overset{!}{=} \int_{-\infty}^{\infty}p\left(x\right)\cdot log\left(\frac{p(x)}{q(x)}\right)dx$$ would mean that $\frac{p(x)}{q(x)} = 1$, which I assume is not true in general.
Asked
Active
Viewed 28 times
1 Answers
2
There's a misprint in definitions.
In fact by definition $$D_{f}(p,q) = \int_{-\infty}^{\infty}q\left(x\right)\cdot f\left(\frac{p(x)}{q(x)}\right)d\lambda(x)$$ and if $f = x \cdot log(x)$ then $$D_{f} = \int_{-\infty}^{\infty}q\left(x\right)\cdot f\left(\frac{p(x)}{q(x)}\right)d\lambda(x) = \int_{-\infty}^{\infty}q\left(x\right)\cdot \frac{p(x)}{q(x)} \cdot log\left(\frac{p(x)}{q(x)}\right)d\lambda(x) = \int_{-\infty}^{\infty}p\left(x\right)\cdot log\left(\frac{p(x)}{q(x)}\right)d\lambda(x)$$
Sergei Nikolaev
- 580
-
Oh, so simple, thank you! – tintinnabulum Oct 27 '22 at 00:37
-
@tintinnabulum You are welcome! – Sergei Nikolaev Oct 27 '22 at 07:28