3

As we all know that the residuals sum to 0 when there is an intercept. But why the residuals do not sum to 0 when we don't have the intercept in simple linear regression?

1 Answers1

2

Recall in the case with an intercept, we get the least squares regressor by minimizing $$ L=\sum_i(y_i-(b_1x_i+b_0))^2$$ in terms of $b_1 $ and $b_0.$ We can do this in the usual way by taking partial derivatives with respect to $b_0$ and $b_1,$ setting to zero and solving.

The fact that the residuals sum to zero comes directly from setting the partial derivative with respect to the interception coefficient $b_0$ to zero. Taking the derivative with respect to $b_0$ gives $$\frac{\partial L}{\partial b_0}=-2\sum_i(y_i- (b_1x_i+b_0)).$$ But the residuals are exactly $$\hat \epsilon_i = (y_i-(\hat b_1 x_i+\hat b_0)) $$ so setting the derivative to zero gives the condition $$\sum_i \hat \epsilon_i=0.$$

Notice that when you take the derivative with respect to $b_1$ you get a factor of $x_i$ out front, so it's a different kind of condition and not one that guarantees the residuals sum to zero.

Also, there was nothing specific to simple linear regression here, this generalizes immediately to the case where there are multiple regressors. The only important thing is that there is a constant term whose minimization equation gives us the condition. If there is no constant term, there is no such condition and thus no guarantee that the residuals sum to zero.

  • I am sorry, what do you mean "there is a constant term whose minimization equation gives us the condition". I know that you use least square estimation to get the sum of residuals equals zero, right? – Fracblood Oct 01 '17 at 01:51
  • 2
    @Fracblood What I show in this answer is that the fact that the residuals sum to zero follows directly from $\frac{\partial L}{\partial b_0}=0$ where $L$ is the sum of square error and $b_0$ is the constant term. If there is no constant term this equation is not part of your minimization problem. – spaceisdarkgreen Oct 01 '17 at 01:54