2

I am attempting to find the mathematical representation of a surface given a set of (x,y,z) data points. I recently tried using the method of least-squares which worked well for most of my situations. However, in some instances I am getting the result that in: $(A^{T}*A)^{-1}*A^{T}*b$ the term $(A^{T}*A)$ is ill-conditioned or nearly singular. My formulation for A is in the format $1+x+y+xy+x^{2}+y^{2}$. Is there a way to prevent these instances from happening? I've heard of applying a svd to least-squares as an alternative, but was unable to find exactly how. Any help would be greatly appreciated.

Travis
  • 425
  • What software are you using? My understanding is that solving a poorly conditioned LSQ problem should not involve you inverting the matrix $(A^TA)$; the LSQ solver should take care of things in a different way. –  Jul 01 '14 at 23:29
  • I'm using MATLAB and Fortran, but the language shouldn't affect the formulation. When I looked up the linear least squares method that was how I found to solve for the coefficient or 'least squares estimate' solution. – Travis Jul 02 '14 at 16:52

1 Answers1

0

Try using Moore-Penrose pseudoinverse $A^+\!$, it coincides with $(A^TA)^{-1}A^T$ when the inverse exists but is defined for any $A$. When $A^TA$ is singular there are multiple least square fits, and pseudoinverse will give you the one with the smallest Euclidean norm. Stable numerical methods for computing $A^+$ are based on singular value decomposition or the QR method, but if you are using MATLAB or something similar there should be a direct command for it.

Conifold
  • 11,756
  • From what I have read this sounds like what I am looking for. Just as a sanity would the following be how I formulate a solution? Ax = y, performing svd on A gives [u,s,v], y_hat = transpose(u)*y, x_hat = y_hat/sigma, where sigma is the diagonal of s – Travis Jul 02 '14 at 17:26
  • You'll have to set a tolerance level and only invert diagonal elements of $S$ if they are bigger than that. The rest are treated as zeros and they stay zeros for $A^+$, so there will be an if then for your $x$_hat. And I think you are multiplying by $U^*$ on the right. – Conifold Jul 02 '14 at 20:54