7

The normalizing factor for the product of two multivariate Gaussian densities, $f(x)$ and $g(x)$ with mean vectors $a$ and $b$ respectively, and covariance matrices $A$ and $B$ respectively, is itself a Gaussian with exponent:

$-0.5(a-b)^T(A+B)^{-1}(a-b)$

i.e. the normalizing factor $c$ is:

$c=\cfrac{1}{\sqrt{2\pi\begin{vmatrix}A+B\end{vmatrix}}}e^{-\cfrac{1}{2}(a-b)^T(A+B)^{-1}(a-b)}$

At first sight, I'd hence interpret the normalizing factor as the value of a (hypothetical) density function with mean $b$ and covariance matrix $A+B$, evaluated at $a$.

Consider now an example, where $f(x)$ and $g(x)$ represent densities of two different statisticians' estimates for $x$. Statistician $A$ whose estimate has density $f(x)$ decides to update his estimate, after he saw the estimate for $x$ from the other statistician, named $B$: $x_b$.

$A's$ new estimate, i.e. an estimate conditional on $x_b$ has the density:

$f(x_a|x_b)$.

Now $B$ would in turn use $A's$ initial estimate to update her own estimate.

The new estimate of $B$, conditional on the estimate of $A$ would have the density:

$g(x_b|x_a)$.

Applying the Bayes rule for densities:

$g(x_b|x_a)=\cfrac{f(x_a|x_b)g(x_b)}{f(x_a)}$

So here, the normalizing factor is (also) $f(x_a)$ the unconditional (marginal) density of $x_a$.

Unfortunately, I cannot see the link between $-0.5(a-b)^T(A+B)^{-1}(a-b)$ and that marginal density. What am I missing?

P.S.: how to derive the expression above is indicated in the answer to the question given here: Product of two multivariate Gaussian pdfs - normalizing constant - where online references are given as well.

Community
  • 1
user70160
  • 555
  • What do you mean by "the normalizing factor" here? Do you mean the quantity $C = \int \exp[- (\mathbf{x} - \mathbf{a})^T \mathbf{A}^{-1} (\mathbf{x}-\mathbf{a}) / 2]\ \exp[- (\mathbf{y} - \mathbf{b})^T \mathbf{B}^{-1} (\mathbf{y} - \mathbf{b})/ 2] d\mathbf{x}d\mathbf{y}$ ? – jgyou May 13 '15 at 18:47
  • I have also edited the question above to give the complete factor, not only the exponent as before. – user70160 May 14 '15 at 03:50
  • You'd need to show that the conditional densities are compatible. I would expect this to be true being Gaussians, but it could be not correct. From example 4.3 in that paper, it seems that one requirement if $f(x|y) \propto \textrm{Exp}(\frac{(x-h(y)^2}{2\sigma^2})$, where $h(y)$ is some function of $y$, is that $h(y)$ be a first order polynomial; which is true in your setting because $h(y) = y$. There's another assumption, though, which seems to indicate that the variances have to be the same. – Ryan Warnick Mar 05 '18 at 07:21
  • Another option is to specify $h(y) = a+by$, and adjust $b$ to adjust for the corresponding variance of the estimate of $y$. – Ryan Warnick Mar 05 '18 at 07:24

0 Answers0