48

Where in the history of linear algebra did we pick up on referring to invertible matrices as 'non-singular'? In fact, since

  • the null space of an invertible matrix has a single vector
  • an invertible matrix has a single solution for every possible $b$ in $AX=b$

it's easy to imagine that that invertible matrices would be called 'singular'. What gives?

gebruiker
  • 6,154
Ryan Kohl
  • 491

2 Answers2

57

If you take an $n\times n$ matrix "at random" (you have to make this very precise, but it can be done sensibly), then it will almost certainly be invertible. That is, the generic case is that of an invertible matrix, the special case is that of a matrix that is not invertible.

For example, a $1\times 1$ matrix (with real coefficients) is invertible if and only if it is not the $0$ matrix; for $2\times 2$ matrices, it is invertible if and only if the two rows do not lie in the same line through the origin; for $3\times 3$, if and only if the three rows do not lie in the same plane through the origin; etc.

So here, "singular" is not being taken in the sense of "single", but rather in the sense of "special", "not common". See the dictionary definition: it includes "odd", "exceptional", "unusual", "peculiar".

The noninvertible case is the "special", "uncommon" case for matrices. It is also "singular" in the sense of being the "troublesome" case (you probably know by now that when you are working with matrices, the invertible case is usually the easy one).

Arturo Magidin
  • 398,050
  • 14
    Interesting. I always assumed that a singular matrix was called such because it didn't have a partner! – JavaMan Jun 01 '11 at 20:26
  • @DJC: Whatever helps you remember works! – Arturo Magidin Jun 01 '11 at 20:27
  • 1
    Just to slightly nitpick, if a $3\times 3$ matrix is made of the same row three times, then the rows do not lie in a single plane, but rather on a single line (and infinitely many planes). You might want to choose better words to give out your exact meaning there. – Asaf Karagila Jun 01 '11 at 20:28
  • 3
    @Asaf Karagila: If the three points lie on a single line, then there is certanly a plane (in fact, infinitely many planes) that contain all three points, no? But I'll reword slightly. – Arturo Magidin Jun 01 '11 at 20:29
  • @Arturo: Of course, but the word single somehow gives a sense of uniqueness (and it is unique if indeed only two are on the same line...), won't you agree? Anyhow, it is much clearer now :-) – Asaf Karagila Jun 01 '11 at 20:32
  • Quite interesting - I'd forgotten that connotation. Maybe it's an older term, from back in the days when medicine was expected to cure singularities. – Ryan Kohl Jun 01 '11 at 20:43
  • 8
    The term "singularity" for a place where a function blows up, often because you can't divide by $0$, is much older than "singular" for matrices. Maybe there is an analogical connection. – André Nicolas Jun 01 '11 at 21:24
  • I thought it was something to do with the fact that, viewed as linear transformations, noninvertible matrices have some kind of 'singularity' in the domain $\mathbb{R}^n$. – Sputnik Jun 24 '11 at 01:19
  • The first time I heard the word "singular" used in this way in everyday english was by Holmes in "The Adventures of Sherlock Holmes". It took me quite a while to make the connection. – Matthew Drury Jan 23 '16 at 22:45
4

I don't know for sure the source of the term "singular", but one reason to call matrices singular when the determinant vanishes, is that curves and surfaces and manifolds have singularities, that is, non-smooth points, when the Jacobian matrix of its coordinate functions is a non-invertible matrix. Smooth points are generic, and non-smooth points are very special, singular, according to Sard's lemma.

Another closely related reason to call the singular matrices "singular" is that they are themselves the singular locus of the algebraic variety of matrices up to some fixed rank, which is known as a determinantal variety. So calling them "singular" seems quite natural.

In response to comments, let me flesh out this last justification. If $M(m, n)$ is the set of all $m\times n$ matrices, then let $M_{\leq r}\subseteq M(m, n)$ be the matrices of rank at most $r$, for $r<\text{min}(m,n).$ Then $M_{\leq r}$ is an irreducible variety of codimension $(m-r)(n-r)$. It is defined as the space of matrices for which every order $r+1$ minor determinant vanishes.

Hence $M_{\leq r}$ is the zero locus of the cofactor map or the adjugate map $\text{Adj}_{r+1}\colon M(m, n)\to M\left(\binom{m}{r+1},\binom{n}{r+1}\right),$ which sends a matrix $A$ to the $\binom{m}{r+1}\times\binom{n}{r+1}$ matrix whose $IJ^{\text{th}}$ component, for ordered tuples $I=(i_1,\dotsc,i_{r+1}), 1 \leq i_1 < i_2 < \dotsb < i_r < i_{r+1} \leq m$ and $J=(j_1,\dotsc,j_{r+1}), 1 \leq j_1 < j_2 < \dotsb < j_r < j_{r+1} \leq n$, is the determinant of the $(r+1)\times(r+1)$ matrix whose entries are obtained from $A$ by removing the rows complementary to $I$ and the columns complementary to $J.$ This map is polynomial in the components of the matrix $A$, hence this shows that $M_{\leq r}$ is an algebraic variety.

Recall that the partial derivative of a general $n\times n$ determinant function is

$$ \frac{\partial(\det A)}{\partial A_{ij}}=\text{Adj}_{n-1}(A)_{ij} $$

so the derivative of the $IJ^\text{th}$ component of the map $\text{Adj}_{r+1}\colon M(m, n)\to M\left(\binom{m}{r+1},\binom{n}{r+1}\right)$ is a determinant minor of rank $r$ of $A$. Hence the Jacobian $d(\text{Adj}_{r+1})$ vanishes if and only $\text{Adj}_{r}$ vanishes. In other words, the rank at most $r-1$ matrices are the singular locus of the variety of rank at most $r$ matrices.

For example, the locus of rank at most 1 matrices in $2\times 2$ matrices is those matrices $ \left(\begin{smallmatrix} a & b\\ c & d \end{smallmatrix}\right) $ for which $ad-bc=0$. This is a quadric hypersurface in $\mathbb{A}^4$ of rank 4, which means (as described for example in this answer by Georges Elencwajg) that it is a cone on the Segre embedding $\mathbb{P}^1\times\mathbb{P}^1\to\mathbb{P}^3$, with cone point $A=0.$ That cone point is the sole rank 0 matrix, the singular locus of the quadric hypersurface that is the rank at most 1 matrices.

Note that this doesn't apply when $r=\text{min}(m,n),$ because the set of matrices of rank at most $\text{min}(m,n)$ is all of $M(m, n)=\mathbb{A}^{mn}$ which is nowhere singular. Only the lower than full rank matrices are singular loci. The above argument doesn't apply because there are no $r+1$ order minors when $r=\text{min}(m,n)$. If $\text{min}(m,n)=m$ and $r=m+1$, then $\binom{m}{r+1}=0,$ so $M\left(\binom{m}{r+1},\binom{n}{r+1}\right) = M\left(0,\binom{n}{r+1}\right),$ the empty matrix. This map is constant, hence constant rank (I think the empty matrix has rank zero?), so has no singular points. (Looking for singular points by taking derivatives does not apply because the constant polynomial is not irreducible).

Versions of this fact are also found in the answer here by Georges Elencwaijg and the one here by Qiaochu Yuan for comparison.

ziggurism
  • 16,756
  • I don't understand this answer. The variety of $n \times n$ matrices is isomorphic to $\mathbb{A}^{n^2}$, which has no singular points. – Viktor Vaughn Feb 11 '18 at 22:10
  • @Quasicoherent Yeah now I'm not sure this makes any sense. There is a stratification of $n\times n$ matrices by rank, and a stratification by singular loci. Probably there is a morphism (the singular locus of a stratum is of lower rank), but the full rank matrices have no singular locus... – ziggurism Feb 12 '18 at 19:35
  • Hi @RichardD.James. A recent downvote persuaded me to try to expand my answer. Your comment is addressed in the final paragraph. – ziggurism Apr 03 '20 at 22:36