0

I have the following maximization objective function related to a svm \begin{align} \max_{\gamma,\omega,b} & \ \frac{\hat{\gamma}}{\|\omega\|}\\ \mbox{s.t. } & \ \ \ y^{(i)}(\omega^{T}x^{(i)} + b) \geq \hat{\gamma}, \ \ i = 1, \ldots m \end{align}

Then the author says that:
$\mbox{maximizing } \hat{\gamma}/ \|\omega\| = 1/\|\omega\| $

is the same as minimizing: $\|\omega\|^2$, why is this?

and that our final optimization function is: \begin{align} \max_{\gamma,\omega,b} & \ \frac{1}{2}\|\omega\|^2\\ \mbox{s.t. } & \ \ \ y^{(i)}(\omega^{T}x^{(i)} + b) \geq 1, \ \ i = 1, \ldots m \end{align}

For what I see he considers that $\gamma=1$, but if I replace directly I will end up with: $\max 1/\|\omega\|$, but from where the author raises $\|\omega\|$ to the power of $2$ and divided by $2$. Is he integrating $\|\omega\|$?

Any help?

Kuzja
  • 450
  • 4
  • 12
Lila
  • 479

2 Answers2

1

OK, you want to maximize a function of the form $$ f(x) = 1/g(x) $$ where both $f(x)>0$ and $g(x) > 0$. Well think about the grapg of the function $f(x)=1/x$ for $x>0$. This is a decreasing function, you can se it directly from the graph (draw it!) or by calculating the derivative $f'(x)=-1/x^2$. So, you want $f(x) $ large? then, since it is decreasing in $x$, you want $x$ small, that is, you want to minimize $x$. You can do the generalization yourself.

kjetil b halvorsen
  • 6,625
1

What the author means that you solution is defined up to a multiplier. If a triple $(\gamma,w,b)$ is a solution then for $k>0$, $(k\gamma,kw,kb)$ is also a solution. So assuming that $\gamma>0$ we just can assume that $\gamma=1$. The power 2 appears only to have the problem more analytical. Sum of squares ($||w||^2$) is better than square root sum of squares. I does not affect on solution, since $\min ||w||$ is equivalent $\min ||w||^2$.

Alexander Vigodner
  • 2,237