1

I am working on a proof that $X'XA = X'XB$ if and only if $XA = XB.$ How do I finish this, and then use to prove the result of X (generalized inverse)X = X?

$ X'XA = X'XB $

$ X'XA - X'XB = 0 $

Factor $X'X(A-B) = 0$

Multiply by $(A'-B')$: $(A'-B') X'X(A-B)=(A'-B')0$

Now how to finish?

Then how can I use this result to prove $X(X'X)^{-1}X'X = X$.

majmun
  • 1,464

3 Answers3

1

IFF proofs require proving both directions. Also, I don't think your proof is correct, as what you have just results $0$. Here is my suggestion:

$\rightarrow$ I.e. Forward direction. Assume $X'XA = X'XB$. As you said, $$X'X(A-B) =0 \implies (A-B)'X'X(A-B) = 0 \implies (X(A-B))'X(A-B) =0$$ (why?) From AA^T=0 implies A = 0 we can conclude $$X(A-B) =0$$. From this you can get your result. In case you do not understand the link, basically, if $AA' =0$ then every element of $AA'$ is $0$, which means all the diagonal elements of $AA'$ are $0$, which implies the trace of $AA' =0$, but the $i$th diagonal element of $AA'$ is $x_i'x_i$, where $x_i$ is the $i$th row of $A$. All these elements being zero implies $A =0$ (why? each of these elements is a sum of squares, which implies each element is 0. Think about what equations we have from these diagonal elements).

$\leftarrow$ Reverse Direction. Assume $XA =XB$. What happens if you multiply both sides by $X'$ on the left?

For the second part, note that $(X'X)^{-1}X'X = I \implies X(X'X)^{-1}X'X = XI = X$. What happens if you multiply these by $X'$ on the left? Think about the formula you proved in the first part.

majmun
  • 1,464
1

Let $C = B - A$, so the question is whether $X' X C = 0$ implies $X' X = 0$. I assume your matrices are real (it would be false if you used complex numbers). For any vector $v$,

$$\|X C v\|^2 = v' C' X' X C v $$ If $X'XC = 0$, then $XCv = 0$ for all vectors $v$, so $XC = 0$.

Robert Israel
  • 448,999
1

I'm going to assume that you're using the notation $X'$ to mean the transpose of $X$. You've correctly computed that $(A-B)'X'X(A-B)=0$; in other "words", $Y'Y=0$ where $Y$ is defined to be $X(A-B)$. Your goal is to prove $Y=0$ since that's equivalent to $XA=XB$.

Consider any column vector $V$ of the right length for $YV$ to make sense. Actually, I don't need completely arbitrary such vectors $V$, just the ones consisting of a 1 in one component and 0's in all the others. If the 1 is in position $j$, then the product $YV$ is the $j$'th column of $Y$. So it suffices to show that the products $YV$ are 0.

Well, the square of the length of the vector $YV$ is the inner product of $YV$ with itself, namely (the only entry in the $1\times1$ matrix) $(YV)'(YV)=V'Y'YV=0$ (where I used that $Y'Y=0$). So $YV$ is indeed the zero vector, as desired.

Andreas Blass
  • 71,833
  • (I see Robert Israel posted the same answer while I was typing mine.) Another way to proceed, once you have $Y'Y=0$ is to add up the diagonal entries of $Y'Y$. Of course the result is $0$, but it's also the sum of the squares of all the entries of $Y$. So $Y=0$ follows without having to introduce the vectors $V$. – Andreas Blass Jan 18 '16 at 20:38
  • and Robert Israel. Thank you, and if I understand this answer, then $|X C v|^2 = v' C' X' X C v$ is also what I'll use to show the generalized inverse proof that follows, right? – Jennifer Cooke Jan 18 '16 at 20:49