(Geometric) Intuition behind Different Types of Rank 2 Tensor (Specifically Quadratic Forms)

Question

This is essentially a follow-up to this question: Differences between a matrix and a tensor

I think I have a good intuition/idea for the change of basis for a rank-(1,1) tensor ($A\vec{v} = \vec{w}$) via diagonalization/similarity transformations/Jordan normal form, etc.

Now that I am studying the definition of symplectic and Lorentz transformations, which involve transformations of rank-(2,2) tensors (quadratic forms, $\vec{u}^T A \vec{v}$), I don't really understand the geometric intuition for why the basis of a quadratic form changes as $T^TAT$ instead of $T^{-1}AT$ like for a rank-(1,1) tensor. This is making it more difficult for me to understand the type of "invariance" that the definitions of symplectic and Lorentz transformations are supposed to imply.

---

Essentially: why is the change of basis for a quadratic form have the shape $T^T A T$? Is there a good geometric intuition for this/picture to have in mind?

---

Does this have something to do with Sylvester's Law of Inertia or the relationship between quadratic forms and symmetric bilinear forms? http://www.ucl.ac.uk/~ucahaya/ChapterV.pdf

I guess another way to phrase the question is: what is the difference in the geometric intuitions for similarity and congruence (of matrices)? http://www.maths.qmul.ac.uk/~twm/MTH6140/la26.pdf

This is especially confusing considering I was taught that they meant the same thing in 9th grade geometry -- clearly I know very little about this. Admittedly when the matrices $T$ are orthogonal (unitary), the definitions do coincide, but they don't coincide always -- for instance symplectic matrices are NOT always orthogonal, even though they are defined as changes of basis for the matrix $J$: $A^T J A = J$.

Answer 1

Let me first remark that the tensors you consider here are of type $(0,2)$ and not of type $(2,2)$. Initially for such a tensor you have a bilinear form $b:V\times V\to\mathbb R$ then the obvious way how a linear isomorphism $\phi:V\to V$ acts on such a form is via $(v,w)\mapsto b(\phi(v),\phi(w))$. So the conditions you are referring to just express the fact that $b(\phi(v),\phi(w))=b(v,w)$ for all $v,w$ in terms of a matrix representation of $\phi$ with respect to some basis $\mathcal B=\{v_1,\dots,v_n\}$ for $V$. You can either proceed directly by looking at the matrix $(b_{ij})$ defined by $b_{ij}:=b(v_i,v_j)$ and compute to see that you get a transpose into the picture rather than an inverse.

Moreconceptually, you can phrase things in terms of an inner product on $V$ and and orthonormal basis $\mathcal B$ for that inner product: Fixing $v\in V$, the map $w\mapsto b(v,w)$ is a linear functional on $V$, so there is a unique element $f(v)\in V$ such that $b(v,w)=\langle f(v),w\rangle$ for. A short computation then shows that the map $v\mapsto f(v)$ is linear, so $f(v)=Jv$ for some matrix $J$. (Further $J$ is symmetric is $b$ is symmetric and skew symmetric if $b$ is skew symmetric.) But then $b(v,w)=\langle Jv,w\rangle$ and $b(Av,Aw)=\langle JAv,Aw\rangle=\langle A^tJAv,w\rangle$, and these two coincide for all $v,w$ if and only if $A^tJAv=Jv$ for all $v$ and hence if and only if $A^tJA=J$.

I am not sure whether you can hope for "geometric intuition" in this case. It is just the natural way how bilinear forms transform under linear isomorphisms (and in fact the description via matrices is a bit artificial).