1

I have a question regarding the following solution, which seems to be wrong but I don't know why exactly.

Suppose we have the problem:

$$\inf_x \left\{ \frac{1}{2}e\|x\|^2_2-\langle m,x\rangle\right\}$$

for positive scalar $e$ and vectors $x,m\in\mathbb{R}^n$. The $\langle,\rangle$ denotes the inner product. Now taking gradient one sees that the solution needs to satisfy

$$ ex -m =0 \iff x = \frac{1}{e}m$$

I wanted to solve it by writing $x$ as $x=\lambda u$ with a unit vector $u$ and a scalar $\lambda\ge 0$, i.e. direction and length of $x$. Then the above is equivalent to

$$\inf_{\lambda\ge 0}\inf_{u,\|u\|=1} \left\{ \frac{1}{2}e\|\lambda u\|^2_2-\langle m,\lambda u\rangle \right\} =\inf_{\lambda\ge 0}\inf_{u,\|u\|=1} \left\{ \frac{1}{2}e\lambda^2-\langle m,\lambda u\rangle\right\}$$

the infimum over $u$ is attained for $u=m$, hence

$$\inf_{\lambda\ge 0}\frac{1}{2}e\lambda^2-\lambda \|m\|_2$$

since both is positive, $e$ and $\|m\|$ the best I can get is $\lambda = 0$, i.e. the infimum is zero, which is not the same as above. Where is my mistake?

Physor
  • 4,586
swissy
  • 91
  • 1
  • 11
  • Do you have a real or complex inner product ? – Physor Oct 20 '20 at 18:53
  • "since both is positive,", but it is a difference of positive terms, both of which depends on $\lambda$. At the last stage you have simply to treat the expression under the infimum as a function of $\lambda$ and find at which value of $\lambda$ the derivative vanishes. – Physor Oct 21 '20 at 17:19

2 Answers2

2

Completing the square, \begin{align*} \frac{e}{2}\lambda^2 - \lambda \|m\| &= \frac{e}{2}\left(\lambda^2 - \frac{2 \|m\|}{e}\lambda \right)\\ &= \frac{e}{2}\left(\lambda - \frac{\|m\|}{e}\right)^2 - \frac{\|m\|^2}{2 e}. \end{align*}

The vertex of this parabola is located at $\lambda = \|m\|_2 / e$, giving you your minimum.

Sam Freedman
  • 3,989
1

Your mistake is pointed out in the comment on the question. I'll proceed as follows, to minimize $\frac{1}{2}e\lambda^2-\lambda \|m\|_2$, as in my the comment above, take the derivative and see where it vanishes $$ \frac{d}{d\lambda}\left(\frac{1}{2}e\lambda^2-\lambda \|m\|_2\}\right) = e\lambda -\|m\| \overset{!}{=} 0 \implies \lambda = \frac{\|m\|}{e} \implies x = \frac{\|m\|}{e}u = \frac{1}{e}m $$

Another way

On the assumption that the inner product is real $$ \frac{1}{2}e\|x\|^2_2-\langle m,x\rangle = \langle \frac{1}{2}ex,x\rangle -\langle m,x\rangle = \langle \frac{1}{2}ex - m,x\rangle $$ For the infimum one can conclude that $\frac{1}{2}ex - m$ should be a scalar multiple of $x$ and pointing in the opposite direcrtion $$ x = -\lambda(\frac{1}{2}ex - m) \implies x = \frac{\lambda}{\frac{\lambda}{2}e+1} m $$ where $\lambda \ge 0$. One can substitute this value of $x$ in the first expression to get $\lambda$ $$ \frac{1}{2}e\|x\|^2_2-\langle m,x\rangle = -\frac{\lambda}{(\frac{\lambda}{2}e+1)^2}\|m\|^2 \\ $$ Now one has to find $\lambda$ that maximize $\frac{\lambda}{(\frac{\lambda}{2}e+1)^2}$, which turns out to be $\lambda = \frac{2}{e}$. That means \begin{align} \inf_x \left\{\frac{1}{2}e\|x\|^2_2-\langle m,x\rangle\right\} &= -\frac{1}{2e}\|m\|^2,& x &= \frac{1}{e}m \end{align}

Physor
  • 4,586