0

Physicist here trying to understand the equivalent definitions of the operator norm. Please be gentle.

I think I understand the definition of the operator norm $$ \|A\|_{\rm op} = \inf \{c : \|Av\| \leq c\|v\| \textrm{ for all } v \in V \} $$ but I don't see how the following equivalent statements are equal $$ a = \sup\{\|Av\| : v \in V \textrm{ with } \|v\| \leq 1 \} $$ $$ b = \sup\{\|Av\| : v \in V \textrm{ with } \|v\| = 1 \}. $$

In the answers to this question, someone says "notice that $b \leq a$", whilst someone else claims that "$a \geq b$ is easily seen". In each case there is no elaboration on why those statements might be true.

Would be great if someone could please walk me through this in baby steps.

To give you an idea of my level of knowledge: it's taking me way too long to work out what tags to use for this question.

miterhen
  • 103
  • 1
  • 1
    Hi, for the claim "notice that $b \le a$" you just use the fact that $${ ||v||=1} \subset {||v|| \le 1}$$ which means that for $a$ you take the supremum over a bigger set – Lukas May 19 '21 at 12:02
  • They are only equivalent if $V \ne {0}$. – daw May 19 '21 at 12:42
  • @daw you are right, I forgot to mention that. If $V={0}$, every $c \in \mathbb R$ satisfies $||Av|| \le c ||v||$, so $||A||_{op} = - \infty$ (the infimum of the whole line $\mathbb R$). Similarly $b= - \infty$ as the supremum of the empty set is $- \infty$. But in that case we have $a=0$ since $||0|| \le 1$ and $||A \cdot 0|| = 0$. So in that case $a \ne b$. – Lukas May 19 '21 at 13:32
  • @Lukas: Why does the value of the supremum depend on the size of the set you're taking it over? – miterhen May 19 '21 at 14:07
  • @miterhen This is a general fact. Let $M \subseteq N$ be sets and we look at a function defined on $N$. If the supremum is existing, we have $$\sup_{x \in N} f(x) \ge \sup_{x \in M} f(x)$$ because we are "checking" more values in the set $N$ than in the set $M$. That is, every possible candidate for the supremum lying in $M$ also lies in $N$, but potentially $N$ has an "even better candidate". Therefore the supremum can only become higher when we look for candidates on a bigger set. – Lukas May 19 '21 at 19:19
  • @Lukas That's what I've been looking for. Thanks! – miterhen May 20 '21 at 09:15

1 Answers1

1

Let's look first at why we also have $a \le b$. For that take $v \in V$ with $||v|| <1$. Then we have $$||Av|| = || \ ||v|| A \frac{v}{||v||} \ || = ||v|| \cdot ||A \frac{v}{||v||} || < ||A \frac{v}{||v||} ||$$ where $\frac{v}{||v||}$ has norm $1$. Therefore the supremum $a$ can never be reached in a point $v \in V$ with $||v|| <1$. With the fact $a \ge b$ as said in the comments we get $a=b$.

Now: why is $a$ equal to the operator norm? We look at the following $$||Av|| \le c ||v|| \Leftrightarrow \frac{||Av||}{||v||} \le c \Leftrightarrow ||A \frac{v}{||v||} || \le c$$ So when we look for the smallest $c$ satisfying this inequality for every $v$ we can alternatively look at the supremum $$\sup_{ v \in V} \{||A \frac{v}{||v||} \}$$ which is the same as $$a= \sup_{||v||=1} \{||Av||\}$$ because the following two sets are equal: $$\{ \frac{v}{||v||}: v \in V \} = \{v \in V: ||v||=1\}$$ (You can write every vector of norm 1 as any vector divided by its norm)

And all in all this gives us $||A||_{op} = a=b$.

Lukas
  • 2,421
  • Thanks for your answer. I think seeing it written out in words helped a lot, but I'm still struggling with the $b \leq a$ bit. – miterhen May 19 '21 at 14:09