This is sort of a weird question, but I am trying to understand why 90 degrees or pi/2 radians is the angle that corresponds to orthogonality in 3-d (or really any dimension of) Euclidean space. Said another way, what's so special about 1/4 of a complete rotation? Why is that the amount of rotation needed to end up in an orthogonal direction to where you started?
I think this must have a connection to the symmetries of the space but I can't put my finger on how to express it.
In 2D we can use the definition of a circle $\langle X,\,X\rangle = 1$, whence if a point with position vector $X$ moves smoothly on the circle, we get $\langle X,\,\dot{X}\rangle = 0$. With a little bit of work from this beginning, one can now prove that the linear mapping keeping the circle invariant is of the form:
$$R = \left(\begin{array}{cc}c&-s\\s&c\end{array}\right)$$
with $c^s+s^2=1$. Which mapping of this kind maps a vector into an orthogonal vector? In other words, we seek solutions to:
$$(x,\,y)\left(\begin{array}{cc}c&-s\\s&c\end{array}\right)\left(\begin{array}{c}x\\y\end{array}\right) = 0$$
and there are precisely two of them:
$$i=\pm\left(\begin{array}{cc}0&-1\\1&0\end{array}\right)$$
For either of these solutions, we can readily show that $\mathrm{id},\,i,\,i^2,\,i^3$ are all distinct and that $i^4=\mathrm{id}$. So four applications of our operator $i$ must impart exactly one full rotation.
Therefore, the isometry that maps a vector to an orthogonal one is the same as the isometry that corresponds to a quarter turn.
So, in summary, we can see that the answer to your question comes from both Danu's AND WillO's comments:
From Danu:
It's a consequence of the definition of the inner product on $\mathbb{R}^N$.
And from WillO:
It's because $i^4=1$
Further Questions from OP
...The derivation you give shows explicitly why a quarter turn maps a vector to an orthogonal one, but I think you probably couldn't even write down a rotation matrix in that form if the space had a structure in which that was false. ... was thinking about the meaning of orthogonality in Minkowski space and in non-Euclidean spaces that might be relevant for actual physical space.
Think of it this way: the matrix's structure encodes orthogonality: witness that its rows/ columns are orthogonal and it conserves inner products. Indeed in 2D it is the only linear mapping that both does so and preserves sense (reflexions also conserve inner products, but they multiply the angle between the two vectors concerned by -1). So it goes hand in hand with the definition of orthogonality. To understand this, witness that a matrix $M$ conserves inner products in Euclidean space iff
$$X^T\,M\,Y= X^T\,Y =\langle X,\,Y\rangle\,\forall\,X,\,Y\in\mathbb{R}^N\tag{1}$$
and write down the general matrix that does this: you'll find that its a rotation composed with a reflexion (indeed one can begin with the seemingly weaker notion of norm conservation $X^T\,M\,X= X^T\,X\,\forall\,X\in\mathbb{R}^N$, write down this condition for vectors $X$, $Y$ and $X+Y$, combine the resulting equations by dint of billinearity and you can derive (1) above). So the matrix's structure follows from the definition of orthogonality in Euclidean space.
A gross difference in Minkowski space is that the intersection of a vector space and its orthogonal complement is not trivial: there are nonzero null vectors ("lightlike vectors") s.t. $\langle X,\,X\rangle=0$, so you can't split the vector spce into a direct sum of subspace $U$ its orthogonal complement $U^\perp$ since $U\cap U^\perp\neq \{0\}$: the intersection of the two in general holds vectors other than 0.
...It seems like something important about physics would break if rotations worked some other way - that is, if it took 6 rotations by 90 degrees to get back to where you started....
Have you heard of spinors, covers of Lie groups and/or representations of groups? There are representations of rotations that take 8 lots of π/2 to get back to the identity. This is not quite the same thing, but it's intriguing nonetheless. $SU(2)$ is the double cover of $SO(3)$ and so if you represent multiplication by quaternions, quaternions act on vectors in a way that the unit $i$ corresponds to a half turn. You might like to see my answer here for further details. The scale factor here is 2, and $SU(2)$ is the universal cover of $SO(3)$ so all covers of $SU(3)$ must be contained in $SU(2)$ and so you get objects that take two full turns to return to the identity, objects that take one full turn to do so and nothing else: there are no scale factors of 3 or 6 for example.
This may be more simplistic than you require.
You said " ... any dimension..." so start with 2 dimensions.
It seems obvious to me that it you turn through 1/4 of a full rotation in 2 dimensions that you have turned through the angle of orthogonality.
If that is no "obvious" in the context of your question (and it may not be) then the following simplistic answer does not satisfy. Otherwise ...
It's related to the fact that a circle has circumference 2 x Pi x radius (= R) and that a radian is defined as the angle which encompasses an arc of length R. Given that definition the result "just drops out".
ie a full circle = 360 degrees = 2 x Pi radians
So 90 degrees = angle of "orthogonality"
= 360 degrees / 4 = 2 x Pi / 4 = Pi/2
