1

Why is the spectral norm, the largest singular value? I understand the proof behind this, but I can not intuitively explain why the spectral norm is the largest singular value. It makes sense that it is "The maximum 'scale', by which the matrix can 'stretch' a vector" (Meaning of the spectral norm of a matrix), but I am wondering why the largest singular value is what represents this.

Why is it not a combination of all the singular values? I am also wondering how it relates intuitively and geometrically to the Frobenius norm. Thank you for the clarification.

user19402204
  • 41
  • The SVD of a matrix is the representation of that matrix as a rotation, followed by dilations along the coordinate axes by the corresponding singular values, followed by another rotation. Since the rotations don't change length, all of the "stretching" is determined by the singular values, and the largest stretching is the largest singular value. – Damian Pavlyshyn Nov 30 '22 at 22:46
  • So the spectral norm indicates the largest stretching of a certain axis, but it does not say anything about how much in total a matrix stretches a vector? What norm indicates how much in total (across all axes) a matrix stretches a vector? Would this be the Frobenius norm? Also, how do you know which axis is stretched the most,is it somehow indicated by the first right or left singular vector? – user19402204 Nov 30 '22 at 23:22
  • There is a whole family of Matrix norms, called Schatten norms, that take into account all the singular values, and so can be thought of as some form of "average stretching." This includes the Frobenius norm, and also the nuclear norm, which is the sum of the the singular values, which sounds morel like what you're looking for – Damian Pavlyshyn Nov 30 '22 at 23:45
  • By convention, we order the singular values in the SVD in decreasing order, but you can rotate the "direction of maximum stretching" onto any axis if you wish by changing the order of the columns of $V$ in the SVD. – Damian Pavlyshyn Nov 30 '22 at 23:47
  • Thinking of the schatten norms as some form of average stretching is a really nice way to intuitively describe them.

    "You can rotate the direction of maximum stretching" onto any axis if you wish by changing the order of the columns of in the SVD" - I was under the impression that singular values applied to specific pairs of left and right singular vectors. Is this not the case?

     – user19402204 Dec 01 '22 at 00:15

0 Answers0