2

So we have a function $f(x) = g(x)h(y(x))$ that is convex in $y$ which we want to optimize by choosing the appropriate $y(x)$. I have seen the following done in engineering books, but it just looks so cheesy and pitfall-y.

$\frac{\partial f(x)}{\partial [y(x)]} = g(x)h'(y(x)) = 0$. Solve for $y(x)$ and we are done.

But when taking the partial, aren't we assuming $x$ is fixed? But if $x$ is fixed, so is $y(x)$ and so the partial should be zero! Is this all engineering type handwaving? Why does it work at all?

Benjamin Lindqvist
  • 3,112
  • I see no reason whatsoever for this to work. For example, what happens if you take $y(x)=x$ and $g(x) = x^2 + 1$ and $h(y)=1$? – 5xum Feb 10 '16 at 10:14
  • For reference, I've seen this trick pop up in numerical analysis, control theory and wireless communication. Probably among others. – Benjamin Lindqvist Feb 10 '16 at 10:26

2 Answers2

1

Here is some heuristics supporting this "trick":

Assume that $x\mapsto y_0(x)$ is the optimal function, and consider $$F(\epsilon,u, x):=g(x)h\bigl(y_0(x)+\epsilon u(x)\bigr)$$ for small $|\epsilon|$ and an arbitrary, but reasonable function $u$. If $y_0$ is truly optimal we would have $${\partial F(\epsilon,u, x)\over \partial\epsilon}\biggr|_{\epsilon =0}=0$$ for any $u$, and at any $x$ in the considered interval. Now this derivative computes to $$g(x)h'\bigl(y_0(x)\bigr)u(x)\ .$$ This then would suggest that we necessarily have $h'\bigl(y_0(x)\bigr)=0$ for all $x$, or that $y_0(x)\equiv c$ with $h'(c)=0$.

Christian Blatter
  • 226,825
1

The derivative here is not with respect to the variable $y$, it's with respect to the function $y(x)$; this is a functional derivative. That is, what is being varied and optimized is the entire function $x \mapsto y(x)$, and whether the function $y(x)$ is changing has nothing to do with whether $x$ is changing.

Qiaochu Yuan
  • 419,620