Solving least squares equations

Question

I have these equations $\partial Q/ \partial \beta_0 = -2(\sum_{i=1}^n y_i - n\beta_0 -\beta_1\sum_{i=1}^n x_i)=0$

$\partial Q/\partial \beta_1=-2(\sum_{i=1}^nx_iy_i-\beta_0\sum_{i=1}^nx_i-\beta_1\sum_{i=1}^n x_{i}^2)=0$

$Q=\sum_{i=1}^n[y_i-(\beta_0 +\beta_1x_i)]^2$ the $y_i$ are response variable to known $x_i$.

So I rearrange this to solve the equations $n\beta_0+\beta_1\sum_{i=1}^nx_i=\sum_{i=1}^ny_i$

and $\beta_0\sum_{i=1}^n x_i+\beta_1\sum_{i=1}^n x_{i}^2=\sum_{i=1}^ny_ix_i$

Which I believe means I want $\begin{pmatrix} n&\sum x_i\\\sum x_i&\sum x_{i}^2\\\end{pmatrix}A=\begin{pmatrix} \sum y_i\\\sum y_ix_i\\\end{pmatrix}$

The solutions I should get are $\beta_1 = \frac{\sum_{i=1}^n (x_i-\bar{x})(y_i-\bar{y})}{\sum_{i=1}^n(x_i-\bar{x})^2}$ and $\beta_0=\bar{y}-\beta_1\bar{x}$

I tried computing the inverse matrix to the matrix left multiplied with A and got $\begin{pmatrix} 1+\frac{\bar{x}\sum x_i}{\sum x_{i}^2 -\bar{x}\sum x_i} & \frac{-\bar{x}}{\sum x_{i}^2 - \bar{x}\sum x_i}\\ \frac{-\sum x_i}{\sum x_{i}^2 - \bar{x}\sum x_i} & \frac{1}{\sum x_{i}^2 -\bar{x}\sum x_{i}}\\\end{pmatrix}$

However when I tried left multiplying this to my original equation to find $A$ I couldn't seem to get the intended solution. I can't see where I made a computation error but is this approach correct?

Answer 1

Your inverse is calculated incorrectly. Using the formula $$ \begin{pmatrix}a&b\\c&d\\\end{pmatrix}^{-1}=\frac1{ad-bc}\begin{pmatrix}d&-b\\-c&a\\\end{pmatrix} $$ you should get an inverse of $$ \frac1{n\sum x_i^2-(\sum x_i)^2}\begin{pmatrix}\sum x_i^2&-\sum x_i\\ -\sum x_i&n\\\end{pmatrix} $$ which you then multiply on the left against $\begin{pmatrix}\sum y_i\\\sum y_ix_i\\\end{pmatrix}$. The result will be a $2\times 1$ column vector $\begin{pmatrix}\beta_0\\\beta_1\\\end{pmatrix}$whose second member is $$\beta_1 =\frac{n\sum y_ix_i-\sum x_i\sum y_i}{n\sum x_i^2-(\sum x_i)^2} =\frac{\sum y_ix_i-n\bar x\bar y}{\sum x_i^2-n\bar x^2}$$ which simplifies to the desired formula for $\beta_1$: see this answer or this answer. To get the formula for $\beta_0$, just solve your first equation $$n\beta_0+\beta_1\sum_{i=1}^nx_i=\sum_{i=1}^ny_i$$ for $\beta_0$.