Can eigenvalues be interpreted as coordinates in a vector space of projection operators?

Question

Consider the vector space $\mathbb R^n$. Let $T : \mathbb R^n \to \mathbb R^n$ be a linear operator, and let $\mathbf T$ be its matrix representation with respect to the standard basis in $\mathbb R^n$. Suppose that the eigendecomposition of $\mathbf T$ is \begin{align} \mathbf T &= \mathbf Q \mathbf \Lambda \mathbf Q^{-1} \\ &= \lambda_1 \mathbf v_1 \mathbf u_1^T + \lambda_2 \mathbf v_2 \mathbf u_2^T + \cdots + \lambda_n \mathbf v_n \mathbf u_n^T \end{align} where $\lambda_i$ is the $i$th eigenvalue of $\mathbf{T}$, $\mathbf{v}_i$ is the $i$th column in $\mathbf Q$, and $\mathbf{u}_i^T$ is the $i$th row in $\mathbf Q^{-1}$. The outer products $\mathbf v_i \mathbf u_i^T$ represent "scaled" projection operators, such that if each of them was replaced with $\mathbf v_i \mathbf u_i^T / \mathbf u_i^T \mathbf v_i$, then they would be true projection operators.

I am not sure if the set of all projection operators on $\mathbb R^n$ forms a vector space, since this set is not a subspace of the vector space of all linear operators on $\mathbb R^n$ (not closed under scalar multiplication). However, assuming that the set of all projection operators is indeed a vector space, wouldn't the outer products $\mathbf v_1 \mathbf u_1^T,\mathbf v_2 \mathbf u_2^T,\dots,\mathbf v_n \mathbf u_n^T$ form a basis for this vector space? Furthermore, since $\mathbf T$ is expressed as a linear combination of basis vectors, then wouldn't the eigenvalues be coordinates in this vector space? Essentially, what I am asking is: do linear operators on $\mathbb R^n$ live in a vector space of projections, and are they "located" by their eigenvalues in this vector space?

Answer 1

First, the inner product $\mathbf{u}^\top_i \mathbf{v}_i$ is automatically $1$, by the vectors' construction. Think about how we compute $Q^{-1}Q$: by taking dot products of rows of $Q^{-1}$ and columns of $Q$. This implies that $$\mathbf{u}^\top_i \mathbf{v}_j = I_{ij} = \delta_{ij},$$ i.e. $1$ when $i = j$ and $0$ otherwise. So, the outer products $\mathbf{v}_i\mathbf{u}_i^\top$ are already projections.

But, as for your main question, no, projections are not a vector space, or at least, not obviously a vector space. As you pointed out, they are not a subspace of the space of linear operators. Indeed, your argument starts with an arbitrary diagonalisable operator $\mathbf{T}$, and produces a linear combination of projections. Since not every diagonalisable operator is a projection, this shows that the projections are not a subspace.

If you equip different operations, the set of projections might become a vector space, but if you keep the same addition and scalar multiplication operations that you've been using, then they are definitely not a vector space.

What you have shown is that the projections span the linear operators on $\Bbb{R}^n$. Well, to be fair, not quite: the assumption of an eigendecomposition implies $\mathbf{T}$ is diagonalisable, so you've shown that every diagonalisable operator lies in the span of the projections. You then need another argument for operators that aren't diagonalisable.

You could argue, for example, that if you put any non-diagonalisable argument into Jordan Normal Form, you can add a diagonal (and hence diagonalisable) matrix to it in order to make the matrix diagonalisable. All you need to do is add numbers to the diagonal of the Jordan blocks to make all the eigenvalues distinct. This will produce an upper-triangular matrix with distinct eigenvalues, which must be diagonalisable. This makes every non-diagonalisable operator a difference of two diagonalisable operators, both of which are in the span of the projections. Thus, the projections span every operator.

Answer 2

In general, projection operators do not form a vector space because $P$ is a projection if and only if $f(P)=0$ where $f(t) = t^2-t$. Indeed $P$ is a projection precisely when it acts as the identity on its image, and this is precisely the condition that $P^2 =P$, or $f(P)=0$. But clearly if $P_1$ and $P_2$ satisfy $f$, their sum will only if $P_1P_2+P_2P_1=0$, which will fail in general.

What is true, however, is that if $T$ is diagonalisable, then the algebra generated by $T$ inside $\text{Mat}_n(\mathbb R)$ -- that is, all operators of the form $p(T)$ where $p$ is a polynomial -- contains the projection operators to the eigenspaces of $T$. This is because $T$ is diagonalisable precisely when it satisfies $m(T)=0$ where $m(t)$ is the polynomial $(t-\lambda_1)\ldots(t-\lambda_k)$ where the $\{\lambda_i: 1\leq i \leq k\}$ is the set of eigenvalues of $T$.

Indeed if $T$ is diagonalisable, say $T=Q\Lambda Q^{-1}$ where $\Lambda = \text{diag}(\mu_1,\ldots,\mu_n)$ then since $m(T)=0$ if and only if $$ 0=Q^{-1}m(T)Q=m(Q^{-1}TQ)=m(\Lambda), $$ it is enough to check $m(\Lambda)=\prod_{i=1}^k(\Lambda-\lambda_i.I)=0$. But all the matrices $\Lambda-\lambda_i.I$ are diagonal, and so the product is just given by the products of the diagonal entries, and for each $\mu_j$ there is some $i$ with $\lambda_i = \mu_j$, that is $\Lambda_{jj}-\lambda_i=0$, hence $m(\Lambda)=0$ as claimed.

On the other hand, if $T$ satisfies $m(T)=0$, then one can in fact show that $T$ is diagonalisable by showing that the projection operators lie in the algebra generated by $T$! The key is the fact that, since $m(t)$ has distinct roots, we can use

Lagrange interpolation: If $\{\lambda_1,\ldots,\lambda_k\}$ is a set of $k$ distinct numbers and we let $s_i(t) = \prod_{j\neq i}\frac{t-\lambda_j}{\lambda_i-\lambda_j}$, then if $f$ is any polynomial of degree at most $k-1$, $$ f(t)= \sum_{i=1}^k f(\lambda_i)s_i. $$

Proof: First note that $s_i(\lambda_j) = 1$ if $i=j$ and $0$ otherwise. It follows that $\{s_i(t): 1\leq i\leq k\}$ are linearly independent, since if $\sum_{i=1}^k a_is_i(t)=0$, then for any $j$, $1\leq j\leq k$, setting $t=\lambda_j$ we obtain $a_j=0$. But the space $\mathcal P_k$ of polynomials of degree at most $k-1$ has dimension $k$ (since it has a basis $\{1,t,\ldots,t^{k-1}\}$) and hence $\{s_i(t):1\leq i \leq k\}$ is a basis of $\mathcal P_k$. It follows that for any $f\in \mathcal P_k$ we may write $f = \sum_{i=1}^k c_is_i(t)$, but then taking $t=\lambda_j$ again we see that $c_j = f(\lambda_j)$ as required.

Now let $P_i = s_i(T)$. We claim $T$ is diagonalizable with the $P_i$ being the projection onto the $\lambda_i$-eigenspace of $T$. To see that $P_i$ is a projection map, notice that $(T-\lambda_i)P_i = (T-\lambda_i)s_i(T)=0$, since $(t-\lambda_i).s_i(t)$ is a scalar multiple of $m(T)$. Thus $\text{im}(P_i)\subseteq E_{\lambda_i}$. But if $v \in E_{\lambda_i}$ so that $T(v)=\lambda_i.v$, then $s_i(T)(v) = s_i(\lambda_i).v=1.v$, hence $P_i(v)=v$ for all $v \in E_{\lambda_i}$. It follows that $P_i^2=P_i$ and $\text{im}(P_i)=E_{\lambda_i}$, that is, $P_i$ is a projection map with image the $E_{\lambda_i}$.

Finally, since $I_n = \sum_{i=1}^k s_i(T)=\sum_{i=1}^k P_i$, for any $v \in \mathbb R^n$, $v =\sum_{i=1}^k P_i(v)$ it follows that $\mathbb R^n = \sum_{i=1}^k E_{\lambda_i}$ and since eigenspaces always form a direct sum, $\mathbb R^n = \bigoplus_{i=1}^k E_{\lambda_i}$ so that $T$ is diagonalizable as claimed.