0

Now I am making Almon model. Lag is 3, and polynomial of 2 degree, so I have following linear regression equation $y_{t}$ = $a$ + $c_{0}$$z_{0}$+ $c_{1}$$z_{1}$+$c_{2}$$z_{2}$. I have a list of $y_{t}$, $z_{0}$, $z_{1}$, $z_{2}$ values, how can I apply method least squares to calculate $c$'s?

This method pointed in a book I am reading. I only know how to use this method with models like $P_{m}(x) = a + a_{1}x + a_{2}x^2 + a_{3}x^3+...+a_{m}x^m$

Alexander Kozachenko
  • 115

2 Answers2

0

We can rewrite the regression equations slightly as $$ z_0 c_0 + z_1 c_1 + z_2 c_2 = y_t - a $$ then arrange them as linear system $$ A x = b $$ with $$ A = (z_0, z_1, z_2) \\ x = (c_0, c_1, c_2)^t \\ b = (y_t-a) $$ where the $z_i$ and $y_t-a$ are column vectors each, having $k$ components if your data list has $k$ lines of such data. $k \ge 3$ would be needed.

Then extend to $$ A^t A x = A^t b $$ then $$ x = (A^t A)^{-1} A^t b \quad (*) $$ is a least square approximation, minimizing $$ A x - b $$ in the Euclidean norm.

Example Calculation

Entering some matrix $A$ and vector $b$ for four lines of data:

octave:1> A = [1, 2, 3; 2, 3, 1; 3, 3, 2; 2, 2, 3] 
A =

   1   2   3
   2   3   1
   3   3   2
   2   2   3

octave:2> b = [1;2;2;1]
b =

   1
   2
   2
   1

Calculating the transposed matrix $A^t$:

octave:3> A'
ans =

   1   2   3   2
   2   3   3   2
   3   1   2   3

Calculating the approximation $x$:

octave:4> x = inv(A'*A) * A' * b
x =

   0.079755
   0.674847
  -0.153374

Checking the error vector and its length:

octave:5> e = A*x - b
e =

  -0.030675
   0.030675
  -0.042945
   0.049080

octave:6> el = norm(e)
el =  0.078326
mvw
  • 34,562
  • my math level is very low, please explain why to add $A^t$ and then remove it? – Alexander Kozachenko Nov 08 '15 at 05:32
  • also, is there any useful article on solving? I have table of 45 values, I have only Kramer method in my mind to solve it. – Alexander Kozachenko Nov 08 '15 at 05:40
  • $A$ is a matrix with $k$ rows and $3$ columns. By multiplying it from the left with the transposed matrix $A^t$, which has $3$ rows and $k$ colums, one gets a quadratic matrix $A^tA$ with $k$ rows and colums, for which a inverse can be calculated. This allows to solve for $x$. BTW it is not added and removed. All three matrices in the above equation $(*)$ for $x$ are needed. – mvw Nov 08 '15 at 05:40
  • @usersample I added an example of the calculation. – mvw Nov 09 '15 at 11:14
  • thank you, how did you get b matrix? – Alexander Kozachenko Nov 10 '15 at 08:11
  • I chose just some random small numbers for it. In your case each would be a difference $y_t-a$. – mvw Nov 10 '15 at 11:28
  • Thank you very much, I finally solved it, but $y_{t}-a$ confused me. $y_{t}$ as b works fine. watched this video https://www.khanacademy.org/math/linear-algebra/alternate_bases/orthogonal_projections/v/linear-algebra-another-least-squares-example – Alexander Kozachenko Nov 10 '15 at 12:57
0

Look up regression analysis, particularly the general linear model (in several variables). There is a huge literature on the subject, including estimating errors of the coefficients.

vonbrand
  • 27,812