Meaning of similar transformations

Question

I am studying similar transformations, and was trying to understand what it means. If you have a linear transformation in given basis, this basis is same for input and output spaces, now when you change the basis, you get a new transformation matrix which is similar to the earlier transformation matrix, say $A$. Am I correct? And when you calculate $P$ and $P^{-1}$ and perform $PAP^{-1}$, the $P^{-1}$ converts the input in first basis to input in second basis. Am I right? If not what is the need for $P^{-1}$?

Answer 1

Note: If you talk about similarity the input space of $A$ is the same as its output space. Otherwise you could not apply $P$ from the left and $P^{-1}$ from the right.

Answer: Yes, you can interpret similarity transformations as a change of basis.

Consider the equation $y = A x$ with a matrix $A\in\mathbb{R}^{n\times n}$, column vectors $x,y\in\mathbb{R}^{n\times 1}$ as variables, and a coordinate transformation $\bar y = P y$, $\bar x = P x$. This leads to the equation \begin{align*} P^{-1}\bar y &= AP^{-1} \bar x\\ \bar y &= \underbrace{PAP^{-1}}_{:=\bar A}\; \bar x \end{align*}

Example: Probably the most important similarity transformation is that one to Jordan normal form.

Eigenvalues of a matrix $A\in\mathbb{R}^{n\times n}$ are the complex numbers $\lambda$ for which the matrix

\begin{align*} \mathbf1\lambda - A \end{align*}

is singular ($\mathbf1$ stands for the identity matrix). The geometrical multiplicity of $\lambda$ is the nullity of the matrix $\mathbf1\lambda-A$, i.e., the dimension of its kernel.

The algebraic multiplicity of $\lambda$ is the nullity of $(\mathbf1\lambda-A)^n$.

Eigenvalues and their geometrical multiplicities are invariant w.r.t. similarity transformations $\bar A = PAP^{-1}$ since for any regular matrix $P$ the matrices $\mathbf1\lambda-A$ and $P(\mathbf1\lambda-A)P^{-1}$ have the same rank and expanding $P(\mathbf1\lambda-A)P^{-1}$ yields

\begin{align*} P(\mathbf1\lambda-A)P^{-1} &= P\mathbf1 P^{-1}\lambda - PAP^{-1} = \mathbf1\lambda - \bar A, \end{align*}

which shows that $\mathbf1\lambda-\bar A$ also has the same rank as $P(\mathbf1\lambda-A)P^{-1}$ and $\mathbf1\lambda-A$. The algebraic multiplicity is also invariant w.r.t. similarity transformation since the rank of $(\mathbf 1\lambda - A)^k$ is also the same as that one of $(\mathbf1\lambda - \bar A)^k$ for any $k\in\mathbb{N}$. This is true for the case $k=1$ and follows for any $k$ by induction from

\begin{align*} P(\mathbf1\lambda - A)^kP^{-1} &= P(\mathbf1\lambda-A)P^{-1}P(\mathbf1\lambda - A)^{k-1}P^{-1} = (\mathbf1\lambda - \bar A)P(\mathbf1\lambda - A)^{k-1}P^{-1}. \end{align*}

For any matrix $A\in\mathbb{R}^{n\times n}$ there is a base of complex generalized eigenvectors $v^{(1)},\ldots,v^{(n)}\in\mathbb{C}^{n}$ such that the coordinatized matrix $\bar A$ w.r.t. this base has the block-diagonal form

\begin{align*} \bar A = \begin{pmatrix} J_1 &\mathbf 0 & \ldots &\mathbf 0\\ \mathbf0 & J_2 & \ddots &\vdots \\ \vdots & \ddots & \ddots &\mathbf 0 \\ \mathbf0 & \ldots &\mathbf0& J_n \end{pmatrix} \end{align*}

where each of the Jordan blocks $J_k$ has the structure

\begin{align*} J_k &= \begin{pmatrix} \lambda_k & 1 & 0 & \ldots & 0 \\ 0 & \ddots & \ddots & \ddots & \vdots \\ \vdots & \ddots & \ddots & \ddots & 0 \\ \vdots & & \ddots & \ddots & 1 \\ 0 & \ldots & \ldots & 0 & \lambda_k \end{pmatrix} \end{align*}

with some $\lambda_k\in\mathbb{C}$ and size $\mu_k\in\mathbb{N}$ $(1\leq \mu_k \leq n)$. These $\lambda_k$ are just the eigenvalues of $\bar A$ and $A$ since the part occupied by $J_k$ in $\bar A$ becomes the singular matrix

\begin{align*} \mathbf1\lambda_k - J_k = \begin{pmatrix} 0 & 1 & 0 & \ldots & 0 \\ \vdots & \ddots & \ddots & \ddots & \vdots \\ \vdots & & \ddots & \ddots & 0 \\ \vdots & & & \ddots & 1 \\ 0 & & \ldots & & 0 \end{pmatrix} \end{align*}

One easily sees that $\mathbf1\lambda_k-J_k$ applied to a vector $v=(v_1,\ldots,v_{\mu_k})^T\in\mathbb{C}^{\mu_k}$ dismisses the first component, shifts $v_2,\ldots,v_{\mu_k}$ one position to the front, and zeros out the last component: $(\mathbf1\lambda_k-J_k)\cdot v = (v_2,\ldots,v_{\mu_k},0)^T$. Any vector $v\in\mathbb{C}^{\mu_k}$ is nullified after $(\mathbf1\lambda_k-J_k)$ is applied $\mu_k$ -times. Therefore, the geometrical multiplicity of and eigenvalue $\lambda_k$ of $\bar A$ is the sum of the sizes of all Jordan-blocks for this eigenvalue.

If all Jordan blocks have size 1 the matrix is diagonalizable and the generalized eigenvectors are just eigenvectors.

Example: In the following example two well motivated consecutive similarity transformation are employed to get to the Jordan normal form.

Two masses freely movable into one direction and interconnected by a spring without the influence of external forces can be described by the equations

\begin{align*} \dot x_1 &= v_1\\ \dot v_1 &= \frac{k}{m_1}(x_2-x_1)\\ \dot x_2 &= v_2\\ \dot v_2 &= \frac{k}{m_2}(x_1-x_2) \end{align*}

or written in matrix form

\begin{align*} \begin{pmatrix} \dot x_1\\ \dot v_1\\ \dot x_2\\ \dot v_2 \end{pmatrix} &= \underbrace{\begin{pmatrix} 0& 1& 0& 0\\ -\frac{k}{m_1}& 0 & \frac{k}{m_1}& 0\\ 0& 0& 0& 1\\ \frac{k}{m_2}& 0 & -\frac{k}{m_2}& 0 \end{pmatrix}}_{=:K} \begin{pmatrix} x_1\\ v_1\\ x_2\\ v_2 \end{pmatrix} \end{align*}

where $K$ is the normalized stiffness matrix (short: stiffness matrix). The motion can be decomposed into that one of the center of mass $\bar x = \frac{m_1 x_1 + x_2 m_2}{m_1 + m_2}$ and a relative displacement $\Delta x = x_1 - x_2$. The full transformation for this decomposition is

\begin{align*} \begin{pmatrix} \bar x\\ \bar v\\ \Delta x\\ \Delta v \end{pmatrix} = \underbrace{\begin{pmatrix} \frac{m_1}{m_\Sigma}& 0 & \frac{m_2}{m_\Sigma} & 0\\ 0 &\frac{m_1}{m_\Sigma}& 0 & \frac{m_2}{m_\Sigma}\\ 1&0&-1&0\\ 0&1&0&-1 \end{pmatrix}}_{=:T} \begin{pmatrix} x_1\\ v_1\\ x_2\\ v_2 \end{pmatrix} \end{align*}

with $m_\Sigma := m_1 + m_2$. Note that for determining the inverse $T^{-1}$ one exploits that the system is composed of two independent subsystems for the locations $x$ and the velocities $v$. These two subsystems have the same transformation matrix $T_x:=\begin{pmatrix}m_1/m_\Sigma & m_2/m_\Sigma\\1& -1\end{pmatrix}$ with the inverse $T_x^{-1}=\begin{pmatrix}1& m_2/m_\Sigma\\ 1& - m_1/m_\Sigma\end{pmatrix}$. Combining the transformation matrices for the subsystems we obtain the transformation matrix for the overall system

\begin{align*} T&:= \begin{pmatrix} \frac{m_1}{m_\Sigma}& 0 & \frac{m_2}{m_\Sigma} & 0\\ 0 &\frac{m_1}{m_\Sigma}& 0 & \frac{m_2}{m_\Sigma}\\ 1&0&-1&0\\ 0&1&0&-1 \end{pmatrix} \end{align*}

The stiffness matrix in the new coordinates becomes

\begin{align*} \bar K_\Delta &:= T K T^{-1}\\ &= \begin{pmatrix} \frac{m_1}{m_\Sigma}& 0 & \frac{m_2}{m_\Sigma} & 0\\ 0 &\frac{m_1}{m_\Sigma}& 0 & \frac{m_2}{m_\Sigma}\\ 1&0&-1&0\\ 0&1&0&-1 \end{pmatrix} \begin{pmatrix} 0& 1& 0& 0\\ -\frac{k}{m_1}& 0 & \frac{k}{m_1}& 0\\ 0& 0& 0& 1\\ \frac{k}{m_2}& 0 & -\frac{k}{m_2}& 0 \end{pmatrix} \begin{pmatrix} 1 & 0 & \frac{m_2}{m_\Sigma} & 0\\ 0 & 1 & 0 & \frac{m_2}{m_\Sigma}\\ 1 & 0 & -\frac{m_1}{m_\Sigma} & 0\\ 0 & 1 & 0 & -\frac{m_1}{m_\Sigma} \end{pmatrix} \\ &= \begin{pmatrix} 0& 1 & 0 & 0\\ 0& 0 & 0 & 0\\ 0& 0 & 0 & 1\\ 0& 0 & -\frac{k}{m_{\rm r}}& 0 \end{pmatrix} \end{align*}

with the reduced mass $m_{\rm r}:=\frac{m_1m_2}{m_1+m_2}$. The corresponding system

\begin{align*} \begin{pmatrix} \dot {\bar x}\\ \dot {\bar v}\\ \Delta \dot x\\ \Delta \dot v \end{pmatrix} &= \begin{pmatrix} 0& 1 & 0 & 0\\ 0& 0 & 0 & 0\\ 0& 0 & 0 & 1\\ 0& 0 & -\frac{k}{m_{\rm r}}& 0 \end{pmatrix} \begin{pmatrix} \bar x\\ \bar v\\ \Delta x\\ \Delta v \end{pmatrix} \end{align*}

is already split into two independent subsystems for the center of mass motion

\begin{align*} \begin{pmatrix} \dot{\bar x} \\ \dot{\bar v} \end{pmatrix} &= \begin{pmatrix} 0&1\\ 0&0 \end{pmatrix} \begin{pmatrix} {\bar x} \\ {\bar v} \end{pmatrix} \end{align*}

and the relative displacement

\begin{align*} \begin{pmatrix} \Delta\dot x\\ \Delta\dot v \end{pmatrix} &= \underbrace{\begin{pmatrix} 0&1\\ -\frac{k}{m_{\rm r}}& 0 \end{pmatrix}}_{=:K_\Delta} \begin{pmatrix} \Delta x\\ \Delta v \end{pmatrix}. \end{align*}

The system matrix for the center of mass motion has already the form of a Jordan block with size 2 and eigenvalue 0. For transforming the second diagonal block $K_\Delta$ to Jordan normal form one solves the eigenvalue equation

\begin{align*} 0 &= \operatorname{det}(\mathbf1\lambda-K_\Delta) = \operatorname{det} \begin{pmatrix} \lambda & -1\\ \frac{k}{m_{\rm r}} & \lambda \end{pmatrix} = \lambda^2 + \frac{k}{m_{\rm r}} \end{align*}

which has the solutions $\lambda_\pm = \pm i\sqrt{\frac{k}{m_{\rm r}}} = \pm i\omega$ with $\omega:= \sqrt{\frac{k}{m_{\rm r}}}$ and the imaginary unit $i=\sqrt{-1}$. The corresponding eigenvectors $u_\pm$ are nontrivial solutions of the singular system

\begin{align*} \left(\mathbf1\lambda_\pm - K_\Delta\right) u_\pm &= \mathbf0\\ \begin{pmatrix} \pm i \omega & -1\\ \omega^2 & \pm i\omega \end{pmatrix} \begin{pmatrix} u_{1\pm}\\ u_{2\pm} \end{pmatrix} &= \begin{pmatrix} 0\\0 \end{pmatrix} \end{align*}

such as

\begin{align*} u_\pm &= \begin{pmatrix} 1\\ \pm i\omega \end{pmatrix}. \end{align*}

The action of the matrix $K_\Delta$ on the eigenvectors $v_\pm$ is just a scaling $K_\Delta u_+ = i\omega\, u_+$, $K_\Delta u_- = -i\omega\, u_-$. If we combine these two equations in the columns of a matrix equation we obtain

\begin{align*} K_\Delta \underbrace{\begin{pmatrix} u_+ & u_- \end{pmatrix}}_{=:T_\Delta} &= \begin{pmatrix} i\omega u_+ & -i\omega u_- \end{pmatrix} = \underbrace{\begin{pmatrix} u_+ & u_- \end{pmatrix}}_{=T_\Delta} \begin{pmatrix} i\omega & 0\\ 0 & -i\omega \end{pmatrix} \end{align*}

and left-multiplication with $T_\Delta^{-1}$ gives the wanted similarity transformation into a diagonal matrix

\begin{align*} T_\Delta^{-1} K_\Delta T_\Delta &= \begin{pmatrix} i\omega & 0\\ 0 & -i\omega \end{pmatrix}. \end{align*}