2

Suppose I have matrices $G,H\in \mathbb{R}^{N\times K}$ and vector $w\in\mathbb{R}^K$. I want to express $$ \sum_{i=1}^N \sum_{j=1}^N \sum_{k=1}^K w_k g_{ik}^2 h_{jk}^2 $$ as a matrix equation as cleanly as possible.

I am trying to do it in terms of trace and ordinary matrix multiplication, avoiding vectors of ones and Hadamard products but don't know if that's possible.

Rodrigo de Azevedo
  • 1
Flash
  • 699
  • The inner two sums look like $GH^T$ where $G_{ij}=g^2_{ij}$ and $H_{ij}=h^2_{ij}$ or something like that. – Gregory Grant Jul 06 '16 at 18:45
  • Taking Gregory Grant's suggestion, write $A = (a_{i,j})$, $B = (b_{i,j})$ and call $W$ the diagonal matrix $W = (w_j \delta_{i,j})$. Then the $i,j$ element of the matrix $D = AWB^T$ is $D_{i,j} = \sum_{k,l} a_{i,k} , W_{k,l} , B_{l,j}$. Because of the diagonal nature of $W$, this is $D_{i,j} = \sum_{k} a_{i,k} , w_{k} , b_{k,j}$. Now you have to put the elements $a_{i,j} = g^2_{i,j}, , b_{i,j} = h^2_{j,i}$. – jim Jul 06 '16 at 19:03
  • Then I would need to sum the elements, so the result is like $\mathbf{1}^TG_2WH_2^T\mathbf{1}$, which is a bit messy looking. – Flash Jul 06 '16 at 19:09

1 Answers1

1

Let $\mathrm g_k$ and $\mathrm h_k$ be the $k$-th columns of $\mathrm G$ and $\mathrm H$, respectively. Hence,

$$\begin{array}{rl} \displaystyle\sum_{i=1}^N \sum_{j=1}^N \sum_{k=1}^K w_k \, g_{ik}^2 \, h_{jk}^2 &= \displaystyle\sum_{k=1}^K w_k \left(\sum_{i=1}^N g_{ik}^2\right) \left(\sum_{j=1}^N h_{jk}^2\right)\\ &= \displaystyle\sum_{k=1}^K w_k \, \|\mathrm g_k\|_2^2 \, \|\mathrm h_k\|_2^2\\ &= \begin{bmatrix} \|\mathrm g_1\|_2^2\\ \|\mathrm g_2\|_2^2\\ \vdots\\ \|\mathrm g_K\|_2^2\end{bmatrix}^T \begin{bmatrix} w_1 & 0 & \dots & 0\\ 0 & w_2 & \dots & 0\\ \vdots & \vdots & \ddots & \vdots\\ 0 & 0 & \dots & w_K\\\end{bmatrix}\begin{bmatrix} \|\mathrm h_1\|_2^2\\ \|\mathrm h_2\|_2^2\\ \vdots\\ \|\mathrm h_K\|_2^2\end{bmatrix}\\\\ &= (\mbox{diag}^{-1} (\mathrm G^T \mathrm G))^T \, \mbox{diag} (\mathrm w) \, \mbox{diag}^{-1} (\mathrm H^T \mathrm H)\end{array}$$

Let $\mathrm e_i$ be the $i$-th vector of the standard basis, and let $\mathrm E_i := \mathrm e_i \mathrm e_i^T$. Given an $n \times n$ matrix $\mathrm A$,

$$\mbox{diag}^{-1} (\mathrm A) = \sum_{i=1}^n a_{ii} \mathrm e_i = \sum_{i=1}^n (\mathrm e_i^T \mathrm A \mathrm e_i) \, \mathrm e_i = \sum_{i=1}^n \mathrm e_i \, (\mathrm e_i^T \mathrm A \mathrm e_i) = \sum_{i=1}^n \mathrm E_i \mathrm A \mathrm e_i = \tilde{\mathrm{E}} \, \mbox{vec} (\mathrm A)$$

where $\tilde{\mathrm{E}} := \begin{bmatrix} \mathrm E_1 & \mathrm E_2 & \cdots & \mathrm E_n\end{bmatrix}$. Thus,

$$\displaystyle\sum_{i=1}^N \sum_{j=1}^N \sum_{k=1}^K w_k \, g_{ik}^2 \, h_{jk}^2 = \mbox{vec}^T (\mathrm G^T \mathrm G) \, (\tilde{\mathrm{E}}^T \mbox{diag} (\mathrm w) \tilde{\mathrm{E}}) \, \mbox{vec} (\mathrm H^T \mathrm H)$$

Rodrigo de Azevedo
  • 1