Why is $(1/n)\sum_{i=1}^{n}x^T_iMx_i=\operatorname{tr}[1/nx^T_ix_iM]$

Question

I am trying to understand the following identity (context Maximum Likelihood Estiation), where $M$ is the inverse of a symmetric matrix:

$$\frac{1}{n}\sum_{i=1}^{n}x^T_iMx_i=\operatorname{tr}\left[\frac{1}{n}x_ix^T_iM\right].$$

I tried this equation with n = 1 and $x \in \mathbb{R}^2$ and M as a 2 by 2 matrix but I dont get the same result. Is this identity false?

EDIT: Corrected the identity, now the small example is correct :).

score 3 · Accepted Answer · answered Jul 25 '16 at 16:05

A general fact about traces of matrix products is that (assuming the matrices are of compatible sizes for multiplication),

$\mbox{tr}(ABC)=\mbox{tr}(CAB)$

This invariance of the trace under cyclic permutation is used a lot in formulation of convex optimization problems.

In your expression, $x_{i}^{T}Mx_{i}=\mbox{tr}(x_{i}^{T}Mx_{i})$, because the trace of a scalar is the scalar itself.

Your expression is very slightly off- you should have $\mbox{tr}(x_{i}x_{i}^{T}M)$ on the right hand side.

1 Answers1