1

I don't understand the following derivation:

$$ e_i = y_i - ax_i - b$$

$$ e_i = (y_i - \bar{y}) - a(x_i - \bar{x}) - (b - \bar{y} + a \bar{x}) $$

I don't really understand what they do and why they do it.

To clarify:

$e_i = y_i - \hat{y}_i$, where $ \hat{y}_i$ is the regression function I believe it's called.

Ylyk Coitus
  • 970
  • 2
  • 10
  • 18

2 Answers2

0

Sorry people, I asked to soon:

$$ e_i = (y_i - \bar{y}) - a(x_i - \bar{x}) - (b - \bar{y} + a \bar{x}) = y_i - \bar{y} - ax_i + a\bar{x} - b + \bar{y} - a\bar{x} = y_i - ax_i - b$$

They just want to rewrite it to $$ e_i = (y_i - \bar{y}) - a(x_i - \bar{x}) - (b - \bar{y} + a \bar{x}) $$ because that is per definition $$ v_i - au_i - (b - \bar{y} + a\bar{x})$$ and that simpler expression will help with finding the regression line.

Ylyk Coitus
  • 970
  • 2
  • 10
  • 18
0

Some of the notation is not fully standard, and in particular, you seem not to be distinguishing between the (unobservable) errors $e_i$ and the (observable) residuals $\hat e_i$.

The usual notation runs like this:

  • $\bar x =$ the average of $x_i$, $i=1,\ldots, n$
  • $\bar y =$ the average of $y_i$, $i=1,\ldots, n$
  • $y_i = ax_i+b+e_i$, where $e_i$ is the $i$th error. The values of $a$ and $b$ are unobservable because you see only the sample $(y_i,x_i)$, $i=1,\ldots,n$ and not the whole population, and $e_i$ is unobservable because $a$ and $b$ are unobservable.
  • $\hat a$ and $\hat b$ are the least-squares estimates of $a$ and $b$. The least-squares estimates are observable because you can compute them base on the sample $(y_i,x_i)$, $i=1,\ldots,n$. They satisfy $\bar y=a\bar x + b$, i.e. the least-squares line passes through the point that is the average of $(y_i,x_i)$, $i=1,\ldots,n$.
  • $\hat y_i = \hat a x_i + \hat b =$ the $i$th "fitted value".
  • $\hat e_i=y_i-\hat y_i=$ the $i$th residual, not to be confused with the $i$th error. The residuals $\hat e_i$ are observable whereas the errors $e_i$ are not. The residuals necsessarily satisfy the two linear constraints $\hat e_1+\cdots+\hat e_n=0$ and $x_1 \hat e_1+\cdots + x_n \hat e_n=0$, whereas the errors, on the other hand, are often taken to be independent.
  • The "regression function" is $x\mapsto y=ax+b$, whereas the fitted value $\hat y_i$ is the value of the regression function when the input is $x_i$.
Michael Hardy
  • 1
  • Hmm, I directly copied this from the sheets I was handed at school. The sheets don't seem too professional and $\hat{e}_i$ isn't mentioned anywhere. What you're saying does make sense though, I need to watch out. – Ylyk Coitus Mar 24 '13 at 13:34
  • What sort of course are you taking and who's teaching it? – Michael Hardy Mar 24 '13 at 13:36
  • Well, I don't like to be an ass but it is an accelerated math class for which students are handpicked. The teacher is our most competent high school math teacher, but I think it's safe to say everybody in that class is smarter than him. He usually prints parts of free university courses and hands them out. This semester it was about linear algebra, linear regression and linear programming. I knew linear algebra from self-studying quantum mechanics, and the other 2 bored me, so I'm ashamed to say I slacked this entire semester, so now I'm catching up. Tl;dr - the class s*cks. – Ylyk Coitus Mar 24 '13 at 13:45
  • It seems I am guilty of an "error". In one of the bullet points I wrote "where $e_i$ is the $i$th residual". I should of course have written "where $e_i$ is the $i$th error". – Michael Hardy Mar 25 '13 at 21:54