6

If we have an orthogonal matrix $U$, then $U^Tx$ is essentially a rotation of the vector $x$. If we have a diagonal matrix $\Lambda$, then $\Lambda x$ is scaling the vector $x$ in each direction by the corresponding diagonal value.

Since any symmetric matrix $S=U\Lambda U^T$, $Sx=U\Lambda U^Tx$ which is rotation of $x$ by some angle $\vartheta$, scaling it by $\Lambda$ and then re-rotating by angle $-\vartheta$. Does this imply that multiplying by any symmetric matrix $S$ is just a scaling by its eigenvalues since the net rotation is $0$?

PinkyWay
  • 4,565
Amrit Prasad
  • 319
  • 1
    Do you possibly mean orthogonal/unitary instead of idempotent? – M. Winter Jun 11 '18 at 21:24
  • 1
    A visualization I found helpful: A unit circle will be transformed into an ellipse with the length of the major axis equal to the larger eigenvalue and the orientation with the corresponding eigenvector, and the minor axis similarly associated with the smaller eigenvalue and and eigenvector. – Doug M Jun 11 '18 at 22:24

2 Answers2

13

Yes, but the orthogonal matrix $U$ determines in which directions this scaling happens. Only applying the diagonal matrix

$$\Lambda=\begin{pmatrix} 2 & 0 \\ 0 & 3 \end{pmatrix}$$

scales by two on the $x$-axis and by three on the $y$-axis.

But you can scale along other (orthogonal) axes if you want to by adding a rotation matrix.

enter image description here

You can imagine this by first transforming the coordinate system by $U^T$, so that the desired stretching axes line up with the coordinate axes (red and blue lines in the image), then you scale along the usual coordinate axes by $\Lambda$, and then you rotate back by $U$.

M. Winter
  • 29,928
  • Does this mean that the non-uniform scaling along the different directions can be interpreted as another rotation? Since the $\Lambda$ mentioned in your example would scale the vector $(x_1,y_1)$ to $(2x_1,3y_1)$ and the latter has a different angle wrt $x$ axis. – Amrit Prasad Jun 11 '18 at 22:38
  • @AmritPrasad I am not sure if I understood your question correctly. A rotation is necessary to non-uniformly stretch along non-coordinate axes, but I would not say that such a transformation can be seen as a rotation (as you cancel it out by applying both, $U$ and $U^T$). – M. Winter Jun 12 '18 at 08:10
  • If we are measuring the angle of a vector $(x_1,y_1)$, then it's going to be $tan^{-1}(\frac{y_1}{x_1})$ wrt to the x-axis. Upon scaling each of these coordinates non-uniformly, the angle is going to become $tan^{-1}(\frac{3y_1}{2x_1})$. This implies a rotation in the original coordinate axes. The only way this wouldn't happen is if $\Lambda=kI$ where $I$ is the identity matrix and $k$ is a constant.

    My point is that just a multiplication by $\Lambda$ can be constituted as a rotation in the sense that the angle defined as $tan^{-1}(y/x)$ is going to change.

     – Amrit Prasad Jun 12 '18 at 19:24
  • 1
    @AmritPrasad I see. Yes, any specific vector will "rotate" (and scale) when we apply any linear transformation. But every vector might rotate differently. If the rotation of all vectors is the same and no scaling happens, then the linear transformation is a rotation (matrix). Otherwise it is not. In the case of $\Lambda$ above, it is no (global) rotation because the vector $(1,0)$ will not change direction, while $(1,1)$ will do so. So the change of angle is not the same for all vectors. – M. Winter Jun 12 '18 at 19:55
  • Yeah. Thanks for your help! Really cleared up a lot of things. – Amrit Prasad Jun 13 '18 at 01:54
  • Hi, did you use Tikz to draw those nice graphs? Could you provide for me the source code? Thanks. – Yes Mar 08 '19 at 05:27
  • @GaryMoore These are made with InkScape. So besides these image files, there is nothing to provide ;). – M. Winter Mar 08 '19 at 09:51
  • @M.Winter Does the scaling along the rotated (orthogonal) axes still hold if the matrix $S$ is not symmetrical? – ecook Mar 14 '20 at 14:21
  • @ecook No. In the non-symmetric case the eigenvalues might not be real (so we scale by a complex number, which is not considered classical scaling), or the matrix is not diagonalizable at all (e.g. shearing). – M. Winter Mar 17 '20 at 20:02
  • @M.Winter Thank you for the answer. BTW, I have another question which I think you may help with. I want to figure out geometrically why a symmetric matrix has real eigenvalues. In other words, I want to know intuitively from the perspective of linear transform that there should be a direction that only scales when transformed by the symmetric matrix. https://math.stackexchange.com/questions/1413763/the-intution-behind-real-symmetric-matrices-and-their-real-eigenvectors gives an answer, but I want to dive more deep. – ecook Mar 18 '20 at 02:19
  • @ecook You should ask this as a separate question. Add a link to the question you mentioned in your comment, and explain why your question is different (otherwise it might be closed as a duplicate). – M. Winter Mar 18 '20 at 12:36
1

General case for a diagonizable matrix

$$S = U.\Lambda.U^{-1} = \begin{pmatrix} 3 & 1 \\ 0 & 2 \end{pmatrix} =\begin{pmatrix} 1 & -1 / \sqrt 2 \\ 0 & 1 / \sqrt 2 \end{pmatrix} \begin{pmatrix} 3 & 0 \\ 0 & 2 \end{pmatrix} \begin{pmatrix} 1 & 1 \\ 0 & \sqrt 2 \end{pmatrix}$$

Multiplying by $S$ is equivalent to multiplying successively by $U^{-1}$, $\Lambda$ and $U$.

  • Matrix $U$ is made of columns which are the eigenvectors of $S$. Multiplying by $U^{-1}$ aligns the eigenvectors with the basis vectors.

  • Scaling by $\Lambda$ preserves the orientation of the eigenvectors, as they are aligned with the basis.

  • The scaled, but with unchanged orientation, eigenvectors are returned to their initial orientation by multiplying by $U$.

enter image description here

The eigenvectors: $$v1=\begin{pmatrix} 1 \\ 0 \end{pmatrix} \text { and } v2=\begin{pmatrix} -1 / \sqrt 2 \\ 1 / \sqrt 2 \end{pmatrix}$$ are shown in color.

Specific case of a symmetric matrix

In the case of a real symmetric matrix $S$, $U$ is orthogonal. $U^{-1}=U^T$ and the relationship can be written:

$$S = U.\Lambda.U^T$$

$U$ is a unitary matrix, in the general case, a rotation matrix.

$$ S = \begin{pmatrix} 3 & \sqrt 2 \\ \sqrt 2 & 2 \end{pmatrix} =\begin{pmatrix} \sqrt {2/3} & -1 / \sqrt 3 \\ 1 / \sqrt 3 & \sqrt {2/3} \end{pmatrix} \begin{pmatrix} 4 & 0 \\ 0 & 1 \end{pmatrix} \begin{pmatrix} \sqrt {2/3} & 1 / \sqrt 3 \\ -1 / \sqrt 3 & \sqrt {2/3} \end{pmatrix} $$

enter image description here

Does this imply that multiplying by any symmetric matrix $S$ is just a scaling by its eigenvalues.

Yes, but as shown, the scaling must be applied in the basis defined by the eigenvectors, or equivalently the basis can be changed prior to apply it. In the case of a real symmetric matrix, the eigenvectors are orthogonal.

mins
  • 395