0

Say, I have $n$-dimensional multivariate Gaussian, $G(x:\mu, \Sigma)$.
($\mu$ is $n$ dimension vector and $\Sigma$ is $n\times n$ matrix.)

Say there is a goal $n$-dimensional vector $a$.

I need to bring and modify multivariate Gaussian close to vector $a$ as much as possible, by updating parameter $\mu$ and $\Sigma$ in $n$-dimensional space.
In other word, I need to minimize $Cost$ function defined as below.

$Cost=E_{x \sim G(x:\mu, \Sigma)}[abs(X-a)]$

Or maybe I should define $Cost$ as squared below.

$Cost=E_{x \sim G(x:\mu, \Sigma)}[(X-a)^2]$

Either way, I need to calculate the derivative of $Cost$ function, which is derivative of expectation w.r.t. $\mu$ and $\Sigma$ as below

$\frac{\partial Cost}{\partial \mu}=?$
$\frac{\partial Cost}{\partial \Sigma}=?$
How can we calculate that?
Thank you.

JimSD
  • 183
  • For the second cost use the fact: $(X-a)^2 = (X-\mu)^2 + 2(X-\mu)(\mu-a) + (\mu-a)^2$. (In 1-d, it is similar in n-d) – user251257 Apr 12 '19 at 07:10

2 Answers2

0

$X-a$ is also a Gaussian vector with mean $\mu - a$ and matrix variance $\Sigma$

So $$\mathbb E \left[\left\|X-a\right\|^2\right] = \left\| \mathbb E \left[X-a\right]\right\|^2 + \text{Var}\left[\left\|X-a\right\|^2\right] = \left\|\mu - a\right\|^2 + \text{Tr}(\Sigma)$$

Now you can differentiate and obtain :

$$\frac\partial{\partial \mu} \mathbb E \left[\left\|X-a\right\|^2\right] = 2(\mu - a)$$

and

$$\frac\partial{\partial \Sigma} \mathbb E \left[\left\|X-a\right\|^2\right] = \text{Tr}(\Sigma) I$$

Kroki
  • 13,135
0

I'd go with the squared error as you can differentiate it. We can compute the Cost function by expanding out the expectations, i.e. $E[X^2] - 2aE[X]+a^2$. Then we know by properties of the Gaussian r.v. what these are in terms of $\mu, \sigma^2$. You can find the derivative of the cost function with respect to a vector, it is the matrix who's rows correspond to differentiating the $i^{th}$ component of the cost function with respect to each of the vectors components, i.e.

The matrix $(\frac{\partial Cost}{\partial \mu})_{ij} = \frac{\partial Cost_i}{\partial \mu_j}$.

You would then set this equal to zero along with the corresponding $\sigma^2$ derivative.

fGDu94
  • 3,916