Average of a set of values using least squares formula

Question

To get the equation of a line $y = ax+b$ passing through a set of $n$ points $(x_i, y_i)$ using least squares formula, we have to solve the following system of linear equations to determine the coefficients, $a$ and $b$: $$\begin{cases} \sum_i{y_ix_i}-a\sum_i{x_i^2}-b\sum_i{x_i} = 0, \\ \sum_i{y_i}-a\sum_i{x_i}-bn= 0 \end{cases} $$ I want to use this formula to find the equation $y = b=\text{constant}$. That is, we substitute $a$ by $0$ in the system above to get: $$ \begin{cases} b = {\sum_i{y_ix_i}\over \sum_i{x_i}} \\ b = {\sum_i{y_i}\over n} \end{cases} $$

The second expression of $b$ is obvious.

Why did we get two expressions for $b$? which one should I use?

PierreCarre · Answer 1 · 2019-03-25T19:11:05.100

Least squares do not work like that. To obtain a least squares approximation (for instance linear regression) you must fix the form of the regression line right from the start. Setting $a=0$ in the formula for linear regression will not get you the formula for the regression with a constant function. This happens because the function to be minimized changes accordingly to the type of regression to be considered. Take for instance the data $(0,0), (1,2), (2,1)$. Least squares approximations by a constant function, a linear function and a quadratic function are given by

$$ y=1, \qquad y = \frac 12 + \frac 12 x, \qquad y = \frac 72 x - \frac 32 x^2, $$

respectively.

score 0 · Answer 2 · answered Mar 25 '19 at 18:57

0

Your mistake is to consider that $a=0$ satisfies the equations. It does not.

You were looking for the minimum of the unexplained error, by canceling the gradient. If you arbitrarily set $a=0$, you are not getting the minimum.

The least-squares fit of the model $y=b$ is straightforward, leading to a single equation in a single unknown.

answered Mar 25 '19 at 18:57

For $y=b$ fit with least squares, you have $\widehat{y}i=b$, so you want to choose $b$ to minimise $\sum\limits{i=1}^{n}(y_i-b)^2$. Differentiate with respect to $b$ and set the derivative to $0$. – Minus One-Twelfth Mar 25 '19 at 19:13
You will need to know some calculus (differentiation). Do you know any calculus? There are other ways to minimise it too, but they involve more "trickery". – Minus One-Twelfth Mar 25 '19 at 19:17
Yes I kno derivatives, but the expression you told me is very complex – IamNotaMathematician Mar 25 '19 at 19:41
@IamNotaMathematician: this is strange, the system you are solving in the post is more complex ! – Mar 25 '19 at 20:50
@YvesDaoust: I just found that system in a book, the, I replaced $a = 0$ – IamNotaMathematician Mar 25 '19 at 21:32

Average of a set of values using least squares formula

2 Answers2