I've been thinking about Householder transformation for the past few days and one point appears to be escaping my insight. I hope that the following description helps someone correct my understanding and point me in the right direction. Based on a book I've been reading, in contrast to the popular convention, I'll assume that all vectors in the following discussion are row vectors.
Let $\textbf{e}_0$ denote the leading row basis vector in $n$-dimensional Euclidean space. The goal is to transform a row vector, $\textbf{z}$ to the form $||\textbf{z}||\textbf{e}_0$ using an orthogonal, involutary matrix, $\Theta$. The vectors, $\textbf{z}$ and $||\textbf{z}||\textbf{e}_0$ are as shown in Figure 1 below.
Suppose now that the difference $\textbf{z}-||\textbf{z}||\textbf{e}_0$ is aligned in the direction of some row vector, $\textbf{g}$. However, we do not know "how much" of $g$ is represented by $\textbf{z}-||\textbf{z}||e_0$. Let's call it $2a\textbf{g}$. This vector is shown in yellow in Figure 2 below.
Since $\textbf{z}$ and $||\textbf{z}||\textbf{e}_0$ are equal in length, they form the sides of an isosceles triangle. A perpendicular from origin to $2a\textbf{g}$ bisects it. Therefore, the vector $(\textbf{z}-a\textbf{g})$ is orthogonal to $\textbf{g}$. That is,
$\begin{eqnarray} &\phantom{\Rightarrow}&(\textbf{z}-a\textbf{g})\textbf{g}^T=0 \\ &\Rightarrow& \textbf{z}\textbf{g}^T-a\textbf{g}\textbf{g}^T=0 \\ &\Rightarrow& a =||\textbf{g}||^{-2}\textbf{z}\textbf{g}^T. \end{eqnarray}$
On the other hand, based on Figure 2, we see that performing the vector addition $(\textbf{z}-2a\textbf{g})$ will take us to $||\textbf{z}||\textbf{e}_0$ which is the reflection of $\textbf{z}$ w.r.t the perpendicular bisector from origin to $2a\textbf{g}$. In other words,
$\begin{eqnarray} &\phantom{\Rightarrow}& ||\textbf{z}||\textbf{e}_0 = \textbf{z}-2a\textbf{g}\\ &\Rightarrow& ||\textbf{z}||\textbf{e}_0 = \textbf{z}-2||\textbf{g}||^{-2}\textbf{z}\textbf{g}^T\textbf{g} \\ &\Rightarrow& ||\textbf{z}||\textbf{e}_0 = \textbf{z}\;(\,\textbf{I}-2||\textbf{g}||^{-2}\textbf{g}^T\textbf{g}). \end{eqnarray}$
From the last equation, we can read off $\Theta=(\,\textbf{I}-2||\textbf{g}||^{-2}\textbf{g}^T\textbf{g})$ as the required orthogonal, involutary matrix that transforms $\textbf{z}$ to the form $||\textbf{z}||\textbf{e}_0$.
Now, an example. Suppose that $\textbf{z} = [1\;\;0.75\;\;0.25]$. Here, $||\textbf{z}||=1.2748$. Hence, the transformation we are looking for is $\textbf{z}_t=||\textbf{z}||\textbf{e}_0 = [1.2748\;\;0\;\;0]$. Let us now write, $\textbf{g}=\textbf{z}-||\textbf{z}||\textbf{e}_0=[-0.2748\;\;0.75\;\;0.25]$. Notice that according to Figure 2, $\textbf{z}-||\textbf{z}||\textbf{e}_0$ should in fact be $2a\textbf{g}$. But, if we do the computation, we'll see that $a=0.5$ regardless of what $\textbf{z}$ is. That is, $\textbf{g}=2a\textbf{g}$. In fact, if we substitute $\textbf{g}=\textbf{z}-||\textbf{z}||\textbf{e}_0$ in $a =||\textbf{g}||^{-2}\textbf{z}\textbf{g}^T$ above, we can show with a little bit of algebra that $a=0.5$.
After finding $\Theta$ by evaluating $(\,\textbf{I}-2||\textbf{g}||^{-2}\textbf{g}^T\textbf{g})$, we get some numbers and we will see that $\textbf{z}\,\Theta=[1.2748\;\;0\;\;0]$. The point that I'm missing is as follows: if we had called the difference, $\textbf{z}-||\textbf{z}||e_0$ in the third paragraph above as simply $\textbf{g}$, we will not be able to find $a$ and hence $\Theta$. But, I know from Figure 2 that I might as well call $\textbf{z}-||\textbf{z}||e_0$ as $\textbf{g}$ and observe that $a$ needs to be $0.5$ based on the properties of an isosceles triangle here. I'm missing some intuition here. Any comments? Thanks.
Update: Can I instead make the following argument to explain how the Householder transformation matrix, $\Theta$ is derived by replacing a portion of the above discussion as follows:
Suppose now that the difference $\textbf{z}-||\textbf{z}||\textbf{e}_0$ is $\textbf{g}$, i.e., $\textbf{g}:=\textbf{z}-||\textbf{z}||\textbf{e}_0$. This vector is shown in yellow in Figure 3 below.
Since $\textbf{z}$ and $||\textbf{z}||\textbf{e}_0$ are equal in length, they form the sides of an isosceles triangle. A perpendicular from origin to $\textbf{g}$ bisects it. Therefore, the vector $(\textbf{z}-0.5\textbf{g})$ is orthogonal to $\textbf{g}$. That is,
$\begin{eqnarray} &\phantom{\Rightarrow}&(\textbf{z}-0.5\textbf{g})\textbf{g}^T=0 \\ &\Rightarrow& \textbf{z}\textbf{g}^T-0.5\textbf{g}\textbf{g}^T=0 \\ &\Rightarrow& 0.5 =||\textbf{g}||^{-2}\textbf{z}\textbf{g}^T. \end{eqnarray}$
On the other hand, based on Figure 3, we see that performing the vector addition $(\textbf{z}-\textbf{g})$ will take us to $||\textbf{z}||\textbf{e}_0$ which is the reflection of $\textbf{z}$ w.r.t the perpendicular bisector from origin to $\textbf{g}$. In other words,
$\begin{eqnarray} &\phantom{\Rightarrow}& ||\textbf{z}||\textbf{e}_0 = \textbf{z}-\textbf{g}\\ &\Rightarrow& ||\textbf{z}||\textbf{e}_0 = \textbf{z}-2\times 0.5\textbf{g}\\ &\Rightarrow& ||\textbf{z}||\textbf{e}_0 = \textbf{z}-2||\textbf{g}||^{-2}\textbf{z}\textbf{g}^T\textbf{g} \;\;\;\;\;\textrm{ (where, we replaced 0.5 with } ||\textbf{g}||^{-2}\textbf{z}\textbf{g}^T \textrm{ from before)}\\ &\Rightarrow& ||\textbf{z}||\textbf{e}_0 = \textbf{z}\;(\,\textbf{I}-2||\textbf{g}||^{-2}\textbf{g}^T\textbf{g}). \end{eqnarray}$
From the last equation, we can read off $\Theta=(\,\textbf{I}-2||\textbf{g}||^{-2}\textbf{g}^T\textbf{g})$ as the required orthogonal, involutary matrix that transforms $\textbf{z}$ to the form $||\textbf{z}||\textbf{e}_0$.
As you can see, this explanation appears very roundabout. Is there an alternative way? Thanks for your comments.
