13

I'm reading a paper regarding the consistency of a statistical estimator, and the author claimed that the following is an identity:

$$ \mathbf{x}^\top (\Sigma + \mathbf{x}\mathbf{x}^\top)^{-1}\mathbf{x} = 1- \frac{\det (\Sigma)}{\det (\Sigma+\mathbf{x}\mathbf{x}^\top)}$$

Here $\mathbf{x}$ is a vector and $\Sigma$ is a covariance matrix. Apart from that, I don't think the author has specified any more assumptions/constraints.

While superficially it appears that this result might be provable via the Sherman-Morrison formula, I haven't been able to work it out.

Any help/pointers would be appreciated ! Thanks.

firdaus
  • 703

4 Answers4

10

The Sherman-Morrison formula states that for an $n \times n$ invertible matrix $A$ and $n \times 1$ vectors $u$ and $v$, we have $(A+uv^T)^{-1} = A^{-1} - \dfrac{A^{-1}uv^T A^{-1}}{1 + v^T A^{-1}u}$.

Also, the matrix determinant lemma states that for an $n \times n$ invertible matrix $A$ and $n \times 1$ vectors $u$ and $v$, we have $\det(A+uv^T) = (1+v^TA^{-1}u)\det(A)$.

Using these two facts, we have

\begin{align*}x^T(\Sigma+xx^T)^{-1}x &= x^T\left(\Sigma^{-1} -\dfrac{\Sigma^{-1}xx^T \Sigma^{-1}}{1 + x^T\Sigma^{-1}x}\right)x \\ &= x^T\Sigma^{-1}x - \dfrac{x^T\Sigma^{-1}xx^T\Sigma^{-1}x}{1+x^T\Sigma^{-1}x} \\ &= x^T\Sigma^{-1}x - \dfrac{(x^T\Sigma^{-1}x)^2}{1+x^T\Sigma^{-1}x} \\ &= \dfrac{x^T\Sigma^{-1}x + (x^T\Sigma^{-1}x)^2}{1+x^T\Sigma^{-1}x} - \dfrac{(x^T\Sigma^{-1}x)^2}{1+x^T\Sigma^{-1}x} \\ &= \dfrac{x^T\Sigma^{-1}x}{1+x^T\Sigma^{-1}x} \\ &= 1 - \dfrac{1}{1+x^T\Sigma^{-1}x} \\ &= 1 - \dfrac{\det(\Sigma)}{(1+x^T\Sigma^{-1}x)\det(\Sigma)} \\ &= 1 - \dfrac{\det (\Sigma)}{\det(\Sigma+xx^T)},\end{align*} as desired.

JimmyK4542
  • 54,331
6

You need two identities to do this, one is the Sherman-Morrison formula $$(A+uv^T)^{-1} = A^{-1} - {A^{-1}uv^T A^{-1} \over 1 + v^T A^{-1}u}$$ and the other is matrix determinant lemma $$\det({A}+{uv}^\mathrm{T}) = (1 + {v}^\mathrm{T}{A}^{-1}{u})\,\det({A})$$ Applying them in that order gives you \begin{align}x^T(\Sigma+xx^T)^{-1}x &= x^T\Sigma^{-1}x - {(x^T\Sigma^{-1}x)^2 \over 1 + x^T \Sigma^{-1}x} \\ &= {x^T\Sigma^{-1}x \over 1+x^T\Sigma^{-1}x } \\ &=1-{1 \over 1 + x^T \Sigma^{-1}x} \\&=1-{\det{(\Sigma)} \over \det{(\Sigma+xx^T)}}\end{align}

Martin Sleziak
  • 53,687
dineshdileep
  • 8,887
4

Whenever a leading/trailing principal submatrix is invertible, the determinant of a matrix is the product of the determinants of that submatrix and of its Schur complement in the enclosing matrix. Let $M=\pmatrix{1&x^T\\ x&\Sigma+xx^T}$. If $\Sigma+xx^T$ is invertible, by considering the Schur complements of respectively $1$ and $\Sigma+xx^T$ in $M$ respectively, we see that $$ \det(\Sigma+xx^T)\det\left(1-x^T(\Sigma+xx^T)^{-1}x\right) =\det(M) =\det(1)\det(\Sigma). $$ Now the result follows immediately.

user1551
  • 139,064
  • I feel silly for not trying to think up a block matrix argument. Excellent! – Ben Grossmann Jan 19 '16 at 11:47
  • This is really nice....I like it because it is self-contained. I believe a variant of proofs of Sherman-Morrison and Matrix-determinant lemma in fact use this kind of block matrix arguments. – dineshdileep Jan 19 '16 at 14:15
1

Here's an incomplete attempt that might help:

Make the substitution $x = \sqrt{\Sigma} y$. We then have $$ \Sigma + xx^T = \sqrt{\Sigma}(I + yy^T)\sqrt{\Sigma} \implies\\ (\Sigma + xx^T)^{-1} = \sqrt{\Sigma}^{-1}(I + yy^T)^{-1}\sqrt{\Sigma}^{-1} $$ With this substitution, the equation becomes $$ y^T(I + yy^T)^{-1}y = 1 - \frac{1}{\det(I + yy^T)} $$ and so we see that it is sufficient to prove this equality.

Now, $$ y^T(I + yy^T)^{-1}y = \\ \operatorname{trace}(y^T(I + yy^T)^{-1}y) =\\ \operatorname{trace}((I + yy^T)^{-1}yy^T) = \\ \operatorname{trace}((I + yy^T)^{-1}(I + yy^T) - (I + yy^T)^{-1}I) = \\ \operatorname{trace}(I) - \operatorname{trace}((I + yy^T)^{-1}) $$

Ben Grossmann
  • 225,327