Is there an intuitive meaning for the spectral norm of a matrix? Why would an algorithm calculate the relative recovery in spectral norm between two images (i.e. one before the algorithm and the other after)? Thanks
Asked
Active
Viewed 5.9k times
2 Answers
72
The spectral norm (also know as Induced 2-norm) is the maximum singular value of a matrix. Intuitively, you can think of it as the maximum 'scale', by which the matrix can 'stretch' a vector.
The maximum singular value is the square root of the maximum eigenvalue or the maximum eigenvalue if the matrix is symmetric/hermitian
-
1thanks - this helps me conceptually. – val Aug 29 '12 at 06:56
-
6This is specifically important since noise will be amplified by this value. For a very nice and intuitive understanding of this I recommend slides 21 and 22 of http://ee263.stanford.edu/lectures.html . – divB May 04 '16 at 07:35
-
@divB, of which lecture? – jds Oct 10 '18 at 00:29
-
327, "SVD and applications" – divB Oct 10 '18 at 22:03
-
2This is only true for symmetric matrices. https://math.stackexchange.com/questions/1437569/do-eigenvectors-correspond-to-direction-of-maximum-scaling#comment7397070_1437569 – information_interchange Mar 28 '20 at 02:09
-
It is also noted there that $A^TA$ provides a symmetric matrix. – ryanwebjackson Dec 13 '21 at 18:59
28
Let us consider the singular value decomposition (SVD) of a matrix $X = U S V^T$, where $U$ and $V$ are matrices containing the left and right singular vectors of $X$ in their columns. $S$ is a diagonal matrix containing the singular values. A intuitive way to think of the norm of $X$ is in terms of the norm of the singular value vector in the diagonal of $S$. This is because the singular values measure the energy of the matrix in various principal directions.
One can now extend the $p$-norm for a finite-dimensional vector to a $m\times n$ matrix by working on this singular value vector:
\begin{align} \|X\|_p &= \left( \sum_{i=1}^{\text{min}(m,n)} \sigma_i^p \right)^{1/p} \end{align}
This is called the Schatten norm of $X$. Specific choices of $p$ yield commonly used matrix norms:
- $p=0$: Gives the rank of the matrix (number of non-zero singular values).
- $p=1$: Gives the nuclear norm (sum of absolute singular values). This is the tightest convex relaxation of the rank.
- $p=2$: Gives the Frobenius norm (square root of the sum of squares of singular values).
- $p=\infty$: Gives the spectral norm (max. singular value).
Kartik Audhkhasi
- 1,496
- 9
- 12
-
1thanks - can we really talk about Schatten p=0 for matrices? since we would be looking at 1/0...? – val Aug 29 '12 at 07:02
-
1Yes we can. And it is equal to the rank of the matrix. Can you clarify your question a bit more? – Kartik Audhkhasi Aug 29 '12 at 13:41
-
2Hi - i was referring to the Schatten norm equation above: the exponent is 1/p. If p=0 we have 1/0. thanks.. p must > 1. – val Aug 29 '12 at 17:31
-
5Think of $p \rightarrow 0$. Then any nonzero singular values will lead to $1$, while $0$ singular values will give $0$ anyway. Hence, you will end up with the number of non-zero singular values. However, for $p \in [0,1]$, the Schatten norm is not a "norm" since it does not satisfy all the properties. However, it is still common practice to call it a "norm". – Kartik Audhkhasi Aug 29 '12 at 18:10
-
-
5I just want to point out the confusion in your notation, the same notation: $||A||_2$ is also being used as spectral norm of a Matrix, which is the $p=\infty$ in your answer. I don't know what is Schatten Norm but one thing is universally agreed is that, matrix is an operator, and its norm should be defined in an operator fashion. – ArtificiallyIntelligent Nov 15 '18 at 19:26
-
Please clear up the notations to be consistent with the literature, wikipedia for example. – QuestionEverything Oct 28 '21 at 23:14