Let $y \sim N(\mu, \Sigma)$
In order to impliment the EM-algorithm I need the expected value:
$$\mathbb{E} \left[ y^T \Sigma^{-1} y \right]$$
Now this is obviously a scalar so I should be able to use linearity of expectation so long as I can express this in a linear sum. But, I am not sure how to express the above as a sum.
If we denote the elements of $\Sigma^{-1}$ as $s_{ij}$, how would I express the above matrix equation as a sum involving $y_i$ and $s_{ij}$?
$ y'Ay $ is quadratic form and as $A$ is (real) symmetric matrix (inverse of the covariance matrix), thus $$ \mathbb{E}[y^TAy] = \mathbb{E}[2\sum_{i>j}a_{ij}y_iy_j+\sum a_{ii}^2 y_i^2] $$ from this step and on it depends on whether $y_i$ and $y_j$ are correlated or not. If not, then $\Sigma$ is diagonal matrix and its inverse is just the reciprocal of $var(y_i)=\sigma_i^2 $, so $$ \mathbb{E}[y^T\Sigma^{-1}y]= 0 + \sum\frac{\sigma_i^2+\mu_i ^ 2}{\sigma_i^2} = n+\sum\frac{\mu_i^2}{\sigma_i^2}, $$ if correlated then instead of $0$ you should add the pairwise covariances, i.e., $$ 2\sum a_{ij} ( \sigma_{ij} + \mu_i\mu_j) $$
It would seem we can generalize this a bit: You can write $y = \mu + \Sigma^{1/2} w$, where $w \sim {\mathcal N}(0_n,I_n)$ is i.i.d. normal and $\Sigma^{1/2}$ is a square-root factor for $\Sigma$ such that $\Sigma^{1/2}(\Sigma^{1/2})^{\rm T} = \Sigma$.
Given that, you can expand $${\mathbb E}\{y^{\rm T} \Sigma^{-1} y\} = \mu^{\rm T} \Sigma^{-1} \mu + {\mathbb E}\{w^{\rm T}\} (\Sigma^{1/2})^{\rm T} \Sigma^{-1} \mu + \mu^{\rm T} \Sigma^{-1} \Sigma^{1/2}{\mathbb E}\{w\} + {\mathbb E}\{w^{\rm T}(\Sigma^{1/2})^{\rm T} \Sigma^{-1} \Sigma^{1/2}w\} \\ = \mu^{\rm T} \Sigma^{-1} \mu + 0 + 0 + {\mathbb E}\{w^{\rm T} w\} = \mu^{\rm T} \Sigma^{-1} \mu + n, $$
where the second and third term are zero since $w$ is zero mean and the third term follows since each element in $w$ has variance 1. Note that this expression holds regardless of whether or not the elements in $y$ are correlated, it is completely general.
