$\mathbb{E}_{x \sim p_{\text{data}}} \left[ \text{KL}(q_{\phi}(z|x) || p_{\theta}(z|x)) \right] = 0$ iff $p_\theta = p_{\text{data}}$

Asked Feb 21 '18 at 06:40

Active Feb 21 '18 at 06:40

Viewed 22 times

Let $p_{\text{data}}$ be the data distribution given and assume $p_\theta, q_\phi$ be parametrized families of distributions.

Claim. $p_\theta(x) = p_{\text{data}}(x)$ for all $x$ iff $$ \mathbb{E}_{x \sim p_{\text{data}}} \left[ \text{KL}(q_{\phi}(z|x) || p_{\theta}(z|x)) \right] = 0. $$

Is this claim true and how can I prove it? I tried it myself but it seems like ($\Leftarrow$) part needs additional assumptions.

asked Feb 21 '18 at 06:40

le4m

3,006

Not sure if I follow your notation. But, KL divergence $ \geq 0$ due to convexity / the gibbs inequality. Therefore, it can only be $0$ when the two distributions are identical. – EDZ Feb 21 '18 at 07:37
@EDZ Yes. So what about it? – le4m Feb 21 '18 at 07:41

$\mathbb{E}_{x \sim p_{\text{data}}} \left[ \text{KL}(q_{\phi}(z|x) || p_{\theta}(z|x)) \right] = 0$ iff $p_\theta = p_{\text{data}}$

0 Answers0