0

I am having trouble using the linear regression normal equations to minimize a function of several variables. The minimization solution is :

$$\beta = (X^TX)^{-1}X^TY$$ where $\beta$ is the coefficient vector of a function to minimize : $$f(x,y)=\beta_1g_1(k_1,x_1,y_1)...\beta_ng_n(k_n,x_1,y_1)$$ $g$ is a function of $k,A \text{ and } B$.

The $k$'s are parameters that I set manually.

$x_i$ and $y_i$ are $i$ measurements taken at points $i\in {1,...,m}$

and The matrix $X$ is

$$X=\begin{bmatrix} g_1(k_1,x_1,y_1) & \dots & g_n(k_n,x_1,y_1) \\ \vdots & & \vdots \\ g_1(k_1,x_m,y_m) & \dots & g_n(k_n,x_m,y_m) \end{bmatrix}$$ and $Y=[Y_{true}(x_1,y_1),...Y_{true}(x_m,y_m)]$, which is a vector of true experimental results at m measurements.

This all works nicely, except when one of the $K=0$. Ideally if I set one of the $K=0$, the corresponding $\beta$ should also equal zero. Instead, setting a $K$ to zero ends up making all of the $\beta$ coefficients equal to zero. Why is this, and is there a work around?

the_candyman
  • 14,064
  • 4
  • 35
  • 62
Frank
  • 880
  • 1
    The formula $\beta = (X^TX)^{-1}X^TY$ only works when $X$ has full column rank. When this is not the case, you need to use pseudo inverses. Serge Lang in his book "Linear Models" has a beautiful chapter on pseudo inverses and the book deals with the general case when $X^TX$ is singular. – Steven Alexis Gregory Jul 07 '18 at 19:15
  • Thanks for your answer. I looked at Serge Lang's list of books but can't find "Linear Models." Do you have a link where I can find out more about the book?

    https://en.wikipedia.org/wiki/Serge_Lang#List_of_books

     – Frank Jul 08 '18 at 07:12
  • According to your $f$, that is $f(x,y) = \prod_{i=1}^n \beta_i g_i(k_i, x_i, y_i)$, hence if $\beta_k=0$ then $f$ becomes $0$. – V. Vancak Jul 08 '18 at 11:28
  • 1
    @Frank - Sorry, I'm getting old.The book is from Searle – Steven Alexis Gregory Jul 08 '18 at 15:52

0 Answers0