Taking the Derivative of the Gaussian Normalizing Condition with respect to $\sigma^2$

Question

In Bishops book, "Statistical Pattern Recognition", there is one exercise, which states to derive the second order moment of the Gaussian Distribution:

$E[x^2] = \int_{-\infty}^{\infty} N(x|\mu, \sigma^2 )x^2dx = \mu^2 + \sigma^2 $

As a hint, the book states to take the derivative of

$\int_{-\infty}^{\infty} N(x|\mu, \sigma^2 )dx = 1 $

with respect to $\sigma^2$.

This is what I have tried and failed miserably and where I need some advice in the right direction. My current attempt:

$ \int_{-\infty}^{\infty} \exp(-\frac{1}{2\sigma^2} (x-\mu)^2)dx = \sqrt{2\pi\sigma^2} $

$ \Longrightarrow \int_{-\infty}^{\infty} \frac{d}{d\sigma^2} \exp(-\frac{1}{2\sigma^2} (x-\mu)^2)dx = \frac{d}{d\sigma^2} \sqrt{2\pi\sigma^2} $

I then tried to some how use the chain rule to compute the right hand side:

$\frac{df}{d\sigma} = \frac{df}{dh} \frac{dh}{d\sigma} $

where $f = \exp(-\frac{1}{2\sigma^2} (x-\mu)^2)$ and $ h = \sigma^2$

This, as least I thought, would allow me to compute $\frac{dh}{d\sigma^2}$ by rearranging:

$\frac{df}{d\sigma^2} = \frac{df}{d\sigma} \div \frac{d\sigma^2}{\sigma} $

$ \Longrightarrow \int_{-\infty}^{\infty} \frac{1}{2\sigma^3} (x-\mu)^2 \exp(-\frac{1}{2z} (x-\mu)^2)dx \div 2\sigma $

$ \Longrightarrow \int_{-\infty}^{\infty} \frac{1}{4\sigma^4} (x-\mu)^2 \exp(-\frac{1}{2z} (x-\mu)^2)dx $

I am sure, that up to this point couple of mistakes already must have happend, which is why I did not write down the right hand side, which derivation is more straight forward.

Answer 1

Note that the hint suggests taking the derivative with respect to $\sigma^2$, not with respect to $\sigma$. If it helps, just replace $\sigma^2$ everywhere with some symbol $z$.

$$\frac{d}{dz} \sqrt{2 \pi z} = \sqrt{\frac{\pi}{2z}}$$ and $$\frac{d}{dz} e^{-\frac{(x-\mu)^2}{2z}} = e^{-\frac{(x-\mu)^2}{2z}} \cdot \frac{(x-\mu)^2}{2} \cdot \frac{1}{z^2}.$$

Thus, $$\frac{1}{2z^2}\int_{-\infty}^\infty (x-\mu)^2 e^{-\frac{(x-\mu)^2}{2z}} \, dx = \sqrt{\frac{\pi}{2z}}.$$ Rearranging yields $$\int_{-\infty}^\infty (x-\mu)^2 \frac{1}{\sqrt{2\pi z}} e^{-\frac{(x-\mu)^2}{2z}} \, dz = \frac{2z^2}{\sqrt{2 \pi z}} \cdot \sqrt{\frac{\pi}{2z}} = z.$$

If you recognize that the left-hand side is $E[(x-\mu)^2]$ then you have $$z = E[(x-\mu)^2] = E[x^2 - 2x\mu + \mu^2] = E[x^2] - 2\mu E[x] + \mu^2 = E[x^2] - \mu^2.$$