Can a non-symmetric projection matrix exist? I am currently using the matrix $M = I - wi^T$ where $i$ is a vector of ones and $w$ is a vector of weights so $w^T i = 1$. Is $M$ a projection matrix? Is it possible for non-symmetric projection matrices to exist (as I believe this matrix to be)? Thanks in advance.
Asked
Active
Viewed 1,886 times
4
-
Very interesting example. Indeed if only $w^Tv=1$ then $(wv^T)(wv^T)=wv^T=P$ and $P$ is projection matrix, then $I-P$ is also the projection matrix. – Widawensen Feb 23 '19 at 12:49
2 Answers
6
Yes and yes.
If by projection matrix you mean that $P^2=P$, then e.g. $$\begin{pmatrix}1&1\\0&0\end{pmatrix}$$ satisfies this.
Your matrix $P=I-wi^T$, when expanded out in components, reads $P_{jk}=\delta_{jk}-w_j i_k$ (using $i$ as a vector is a somewhat unfortunate notation). Then you can check that $\left(P^2\right)_{jk}=P_{jl}P_{lk}=P_{jk}$ indeed holds, by virtue of your condition $w_j i_j=1$.
Update:
Your matrix $P$ acts in the following way:
It annihilates $w$, since $$Pw=\left(I-w i^T\right) w=w-w (i^Tw)=0\,.$$ On the other hand, it projects onto the space orthogonal to $i$, since for any $v$ $$i^T \, Pv=i^T \left(I-w i^T\right) v=i^T v - (i^T w) (i^T v)=0\,.$$
That means it does not project orthogonally, since $Pv\neq P\cdot(v+\lambda i)$ -- rather, it projects 'along $w$', i.e. $Pv=P\cdot(v+\lambda w)$. This bring us back to your first question: $P$ is generically not symmetric, $$P^T=\left(I-w i^T\right)^T=I- i w^T \neq P\,,$$ unless $w=\lambda i$. (The factor lambda is fixed by the normalisation $i^Tw=1$.) That is, excactly when $w$ and $i$ are proportional, $P$ is symmetric and projects orthogonally.
Toffomat
- 2,245
-
So is P*P = P the definition of a projection matrix? It seems like a weak definition! – Christian Jun 01 '17 at 10:54
-
@Christian Indeed, that's the usual definition. The intuition behind it is that $P$ projects the full vector space onto some subspace, but on that subsapce, it doesn't change anything, so once-projected vectors don't get changed by a repeated projection. Your operator e.g. projects onto the space orthogonal to $i$. Of course, an invertible projector is just the identity, so any nontrivial $P$ projects onto a true subspace. (See https://en.wikipedia.org/wiki/Projection_(linear_algebra)). – Toffomat Jun 01 '17 at 11:09
-
Thanks for your help! By the way, a small correction to your last response: my operator e.g projects onto the space orthogonal to w not i... it's basically a generalisation of the matrix M_0 you find in econometrics. It converts a vector into deviation from the weighted sum. As a special case, when each element of w is 1/n then it converts a vector into deviation from mean. I have never seen this matrix in the econometrics literature! – Christian Jun 01 '17 at 13:18
-
@Christian No, the subspace is orthogonal to $i$: Let $v$ be any vector, then (in your notation) $i^T \cdot \left(I-w i^T\right) v=i^Tv - (i^T w) (i^T v)=0$, since $i^T w=1$. So the subspace, i.e. the image of $P$, is orhogonal to $i$, regardless of $w$. However, the image of any given vector $v$ does depend on $w$ -- intuitively, you're projecting on the same subspace, but under different "angles". (Or I have misunderstood your notation, but I guess you want to switch $w$ and $i$ -- all you need here is that $w\cdot i=1$.) – Toffomat Jun 01 '17 at 13:28
-
In other words: you have $i\cot P v=0$ for all $v$, so the projection is onto tha space orthogonal to $i$. However, the projection is "along $w$" - that's why you have $Pw=0$ - so you're not projecting orthogonally onto the image of $P$ (unless $w\sim i$.). – Toffomat Jun 01 '17 at 13:38
-
I meant to say that the matrix $M = I - wi^T$ annihilates vectors that are equal to $\lambda w$ for some number $\lambda$. This matrix converts a vector into deviation from the weighted sum. I don't really understand your argument but i'll take another look. – Christian Jun 01 '17 at 13:45
-
-
Thank you. I think I understand what you mean now. So it's projection but at an angle or along w. That's SO interesting! I didn't realise you could have non-orthogonal or at-an-angle projections! – Christian Jun 01 '17 at 23:53
3
If $P$ is a square matrix such that $P^2=P$ ($P$ is idempotent), then it represents the projection onto $C(P)$ (column space of $P$) along $C(I-P)$ ($I$ the identity matrix), that is, $$ v=Pv+(I-P)v $$ is the unique way to write $v$ as the sum of a vector in $C(P)$ with a vector in $C(I-P)$. Indeed, if $$ v=x+y $$ with $x\in C(P)$ and $y\in C(I-P)$, then $x=Px'$ and $y=(I-P)y'$ for some $x'$ and $y'$, so $$ Pv=P(Px'+(I-P)y')=P^2x'+(P-P^2)y'=Px'+0=x $$ and, similarly, $(I-P)v=y$.
Consequently, $C(P)\cap C(I-P)=\{0\}$.
Conversely, if $U$ is any subspace of $\mathbb{R}^n$ and $\{v_1,v_2,\dots,v_k\}$ is a basis for $U$, if you complete the basis to a basis $\{v_1,\dots,v_k,v_{k+1},\dots,v_n\}$ of $\mathbb{R}^n$, the map $$ v=\alpha_1v_1+\dots+\alpha_kv_k+\alpha_{k+1}v_{k+1}+\dots+\alpha_nv_n \mapsto \alpha_1v_1+\dots+\alpha_kv_k $$ is linear and its matrix $P$ has the property that $P^2=P$.
On the other hand, if you add the assumption that $C(I-P)$ is the orthogonal complement to $C(P)$, then the matrix $P$ is symmetric (where the inner product is the standard one).
Indeed, we must have $\langle (I-P)v,Pw\rangle=0$, for every $v,w$, which translates to $$ ((I-P)v)^TPw=0 $$ that is $$ vPw=vP^TPw $$ for all $v,w$, and so $P=P^TP$, which implies $P^T$ is symmetric.
In your case with $P=I-wi^T$, symmetry reads $$ I-iw^T=I-wi^T $$ that is, $iw^T=wi^T$; multiplying by $i$ on the right, $$ iw^Ti=wi^Ti $$ so $$ w=\frac{\langle w,i\rangle}{\langle i,i\rangle}i $$ and $w$ is a scalar multiple of $i$. Conversely, if $w=\alpha i$, then $I-wi^T=I-\alpha ii^T$ is clearly symmetric.
Hence $I-wi^T$ is symmetric if and only if $w$ is a scalar multiple of $i$. But it is idempotent as soon as $\langle i,w\rangle=1$: $$ (I-wi^T)(I-wi^T)=I-wi^T-wi^T+wi^Twi^T= I-wi^T-wi^T+\langle i,w\rangle wi^T $$
egreg
- 238,574