2

Sorry if this is a complete dummy question, but I haven't done math in years and I'm quite rusty. I'm reading this explanation of least squares regression, which internally uses the quadratic equation solution. I was wondering if someone can tell me if my analysis of this is correct. I'm trying to explain to myself why the expression is clearly minimized...

which is

$$a\left(x+\frac{b}{2a}\right)^2 + \frac{4ac-b^2}{4a}$$

Since the second term does not depend upon x, and the first term is a
times a perfect square (and a > 0), the expression is clearly
minimized when x = -b/(2a).

I suppose this is somewhat equivalent:

$z = a(x+y)^2$ + C

which we can take the derivative of to find the function for the slope of $z$:

$\frac{dx}{dz} z = 2a(x+y)$

set this equal to zero to find the point where the slope is zero (e.g. a minimum or maximum--we know that $a > 0$ so this should be a minimum)

$0 = 2a(x+y)$

$a$ is discarded during the division, and subtract to solve for $x$

$0 = x+y$

$ x =-y$

Glorfindel
  • 3,955
Hamy
  • 123

2 Answers2

2

Differentiating makes things look much more complicated than they are: all we need is some algebra. It is easy to verify (by expanding the right-hand side) that $$ax^2+bx+c=a\left(x+\frac{b}{2a}\right)^2 + \frac{4ac-b^2}{4a}$$ for all $x$. The second term is independent of $x$, it is a constant.

The term $$a\left(x+\frac{b}{2a}\right)^2$$ is the positive $a$ number times a square. A square is never negative, the smallest it can be is $0$. And in this case we can arrange for it to be $0$ by choosing $x=\frac{b}{2a}$.

It follows that the minimum possible value of $ax^2+bx+c$ is $\frac{4ac-b^2}{4a}$, and it is reached at exactly one place, namely $x=-\frac{b}{2a}$.

Remark: All these letters may hide the basic simplicity of the idea. Consider the quadratic $x^2 -6x+17$. This can be rewritten as $(x-3)^2+8$. But $(x-3)^2$ is never negative, so $(x-3)^2+8 \ge 8$ for all $x$. And $(x-3)^2 =0$ at $x=3$, so $(x-3)^2+8$ can be equal to $8$. so the minimum value of $(x-3)^2+8$ is $8$.

André Nicolas
  • 507,029
2

The right side is a constant, so no matter what value of $x$ we choose, it will still be there. If it was a minimum before, it will still be the minimum when we add a constant (consider a curve, then consider shifting the curve $k$ units upwards, clearly it doesn't change the minimum / maximum).

You should realize that we have the square of $x+b/2a$ on the left. The smallest the square of a real number can be is $0$ (at which point the left term will disappear and we will have our minimum). Set the $(x + b/2a)^2$ to 0 and solve to get $x = -b/2a$, which is your solution.

(Another easy way to verify that this is true by considering the derivative of $ax^2 + bx + c$ to get $2ax + b = 0$, and solve for $x = -b/2a$ as the inflection point of the quadratic.)

George V. Williams
  • 5,213
  • Ah, that's what I was not considering -- the constant doesn't change the minimum location, and we don't care about the value at the minimum – Hamy Mar 09 '13 at 01:52