0

I was reading about Support Vector Machines and I found that it's equivalent to solve the problem of maximize this number:

$\frac{1}{\left \| w \right \|}$

with to minimize this number:

$\frac{1}{2}\left \| w \right \|^{2}$

I would like to appreciate any explanation of this fact.

neo33
  • 133

2 Answers2

1

Let's say you have found $\min\frac{1}{2}\left \| w \right \|^{2} = \frac{1}{2}\left \| w_0 \right \|^{2} $.

Then $w_0$ is such that $\left \| w_0 \right \|$ is the smallest.

Because $ x \to \frac{1}{x}$ is a strictly decreasing function on the positive real numbers, it means that $\frac{1}{\left \| w_0 \right \|}$ is the biggest which means $\max\frac{1}{\left \| w \right \|}=\frac{1}{\left \| w_0 \right \|}$.

Dark
  • 1,592
  • Thanks, this really helped me, but the only part that I don´t understand is, why to use the 1/2 in the first equation that could be any number?, and the result are going to be the same?. – neo33 May 09 '16 at 13:39
  • Yes you can replace $\frac{1}{2}$ by any $ \lambda > 0$ and the result would still be the same. I think $\frac{1}{2}$ is here just because it is more convenient when taking the derivative of $w \to w^2$ :) – Dark May 09 '16 at 13:44
  • I agree thanks a lot for the support, i was thinking in the derivative, this was very useful. – neo33 May 09 '16 at 14:13
0

$$ \frac{1}{\|w\|}\le\frac{1}{\|w_0\|}\quad\forall w\Longleftrightarrow\|w_0\|\le\|w\|\quad\forall w\Longleftrightarrow\frac12\,\|w_0\|^2\le\frac12\,\|w\|^2\quad\forall w. $$

Julián Aguirre
  • 76,354
  • Thanks for the support this really helped me, I just have a doubt this result does not depend of the 1/2, that could be any number and the result would be the same? – neo33 May 09 '16 at 13:39