0

Assume V + Ax = b is the equation where V is the vectors of residuals, A is the matrix for coefficients, x is the vector for unknowns, and b is the vector for observation.

It is common to read something like "The least squares estimator is obtained by minimizing V. Therefore we set the partial derivative of V^(t)V with respective to x equal to zero..."

What I don't understand is that by doing so, we may actually find a maximum point for V instead of minimum, since we haven't check for the sign in both direction. Why they can just assume getting a minimum point?

user702371
  • 1
  • Because the linear least squares objective function (which is not the V you show) is convex. – Mark L. Stone Sep 09 '19 at 15:37
  • Thank you for your reply. Do you have any proof by Hessian matrix that proving the second derivative of V^(t) V with respect to X^t is positive defined? I spend a lot of time to find but I couldn't. – user702371 Sep 10 '19 at 16:00
  • https://math.stackexchange.com/questions/483339/proof-of-convexity-of-linear-least-squares – Mark L. Stone Sep 10 '19 at 16:02

0 Answers0