if a scatter plot is above a given equation, is it a bad fit

Question

So, a friend of mine asked for help with a question concerning scatter plots. She graphed it out and looked like

To me, it looked like a good fit. However, other friends said that it was wasn't a good fit because all the data is above the equation and said "you want the data to be evenly spread above and below the line". It has been a while since I've done scatter plots, but I want to know the answer to my dumb question:
If all the given data is above a given model but relatively close to the given model, is it still a bad fit because the data isn't spread evenly above and below the data?

Thank you so much! It's just a dumb question I have and I'm just curious as I haven't done scatter plots in a very long time :)

Answer 1

If all the data is above or below the line of best fit, then the line of the best fit is not the 'best'. What if I increased the $y$-intercept of the line of best fit just slightly? Then, all of the points would be closer to the line, and the line would fit better.

The line of best fit is more properly termed the least squares regression line (LSRL). Let's call this line $y=f(x)$.

The basic idea is that for a given $x-$value, the distance between the point and $(x,f(x))$ should be as small as possible. More precisely, this distance is squared. The squaring means that more weight is given to large discrepancies in distance. Allow me to elaborate:

Let's say you have two points $(x_1,y_1)$ and ($x_2,y_2)$. The distances for each of the points are $|f(x_1)-y_1|$ and $|f(x_2)-y_2|$ respectively. Using these distances, I can come up with a LSRL 'score':

$$ \text{score}=|f(x_1)-y_1|^2+|f(x_2)-y_2|^2 $$

We want this score to be as low as possible. Note that squaring the distances has a nice effect: it means that one discrepancy of $4$ units is valued as worse than two discrepancies of $2$ units as $4^2>2^2+2^2$.

This conforms with our intuition of what the LSRL should look like. Hence, why the LSRL is based on this principle.

Going back to your question, the LSRL is lower when the line is between the dots. You really don't want there to be relatively large distances as in the diagram shown. These large distances would be magnified, giving us a high LSRL 'score'.

On an entirely pragmatic note, if you simply want to find the correct LSRL for the data, then there are many calculators on the web that do this for you—these calculators tell you what the equation of the LSRL is.

Answer 2

Here is a simple linear regression example in which there is a little random normal noise abound the line $y = 2x + 5.$ In R, the regression analysis looks like this:

set.seed(527)
x = runif(20, 0, 10)
y = 2*x + 5 + rnorm(20, 0, 1)
lr.out = lm(y ~ x)
summary(lr.out)
Call:
lm(formula = y ~ x)

Residuals:
     Min       1Q   Median       3Q      Max 
-2.16173 -0.71855  0.03993  0.36845  2.44138 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)   4.9191     0.5092   9.661 1.51e-08 ***
x             1.9479     0.0854  22.809 9.85e-15 ***
---
Signif. codes:  
0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 1.174 on 18 degrees of freedom
Multiple R-squared:  0.9666,    Adjusted R-squared:  0.9647 
F-statistic: 520.3 on 1 and 18 DF,  p-value: 9.851e-15

Briefly, the regression estimates the y-intercept as $b_0 = 4.9191$ and the slope as $b_1=1.9479,$ both of which are very close to the theoretical values $(\beta_0 = 5, \beta_1 = 2.)$

Here is a scatteplot of the $(x,y)$-pairs. The regression line is shown in blue. Typically, the data points will be scattered on either side of the regression line.The theoretical line, used to simulate the data, is shown as a dotted line.

As @Joe (+1) explains, the coefficients $b_0$ and $b_1$ of the regression line are chosen to minimize the sum of squares of vertical distances of the points about the line. These vertical (positive and negative) distances $r_i$ are called residuals. They have $\sum_{i=1}^n r_i = 0.$

plot(x, y, pch=20)
 abline(lr.out, col="blue")
  abline(a = 5, b=2, lty="dotted")