1

Suppose that $X$ is an $m \times n$ matrix and that the matrix $X^TX$ is invertible.

$H = X(X^TX)^{-1}X^T$

where $X^T$ is transpose of $X$; $(X^TX)^{-1}$ is inverse of $X^TX$

we are asked to show that

  1. $\text{tr}(H) = n$.

  2. By considering $H^2v$, where $v$ is an eigenvector of $H$, show that the eigenvalues of $H$ must be either $0$ or $1$.

JimmyK4542
  • 54,331
Samuel Song
  • 115

2 Answers2

1

Hints:

1) Use the fact that $\text{tr}(AB) = \text{tr}(BA)$ and $\text{tr}(I_n) = n$.

2) Note that $H^2 = [X(X^TX)^{-1}X^T][X(X^TX)^{-1}X^T]$ $= X\underbrace{(X^TX)^{-1}(X^TX)}_{I}(X^TX)^{-1}X^T = X(X^TX)^{-1}X^T = H$.

What does this tell you about the eigenvalues of $H$?

JimmyK4542
  • 54,331
  • Thanks for your hint. Here is my way to prove #2. $H^2v = HV = av$ ; $av = H(Hv)$; $av = H(av)$; $av = a(Hv)$; $av = a(av)$; $av = a^2v$; $a=a^2$; Thus, $a=0 or 1$ – Samuel Song Oct 30 '14 at 17:16
0

Note that $X^T v = 0 $ implies $Hv = 0$, and $Hv = 0$ gives $X(X^T X)^{-1} X^T v = 0$, premultiplying by $X^T$ gives $X^T v = 0$. Hence $\ker H = \ker X^T$.

Since $X^T X$ is invertible, we must have $\dim \ker X^T = m-n$. Since $H$ is symmetric, it follows that $H$ has exactly $m-n$ zero eigenvalues.

Now suppose $Hv = \lambda v$. Then $X^T H v = X^T v = \lambda X^T v$. Hence either $X^T v = 0$ (and so $H^T v = 0$) or $\lambda = 1$, from which it follows that the eigenvalues are either 0 or 1. Since $H$ has $m-n$ zero eigenvalues, it has $n$ one eigenvalues and so $\operatorname{tr} H = n$.

copper.hat
  • 172,524