4

Definitions

correlation coefficient $= r = \frac{\sum_{i=1}^{n}(x_i - \bar{x})(y_i - \bar{y})}{\sqrt{\sum_{i=1}^{n}(x_i - \bar{x})^2\sum_{i=1}^{n}(y_i - \bar{y})^2}}$

My Question

What is the motivation of this formula? It's supposed to measure linear relationships on bivariate data, but I don't understand why it would do that as defined. For example, Riemann integrals are said to measure area under a curve, and that makes sense because $\sum f(x_i)\Delta x$ is adding areas of rectangles under the curve $f(x)$ approximating its area more and more as we take more samples. Does such an intuition exist for the correlation coefficient? What is it? My background in statistics is nothing but a bit of discrete probability. I know histograms, data plots, mean, median, range, variance, standard deviation, box plots and scatter plots at this point (from reading the first weeks material on an introductory statistics class).

My Research

All of the "Questions that may already have your answer" seemed to either be asking about what the formula said mathematically or asked questions that were more advanced than my knowledge.

Dunka
  • 2,787
  • 12
  • 41
  • 69

1 Answers1

4

Suppose we have a scatterplot of heights X and weights Y of n subjects. The 'center of the data cloud' is at the point $(\bar X,\,\bar Y)$.

One might expect a positive association between heights and weights. Points above and to the right of the center make a positive contribution to the sum $\sum (X_i -\bar X)(Y_i - \bar Y).$ So also do points below and to the left of center.

Points that might suggest a negative association will be above and to the left of center or below and to the right of center. For them, the product $(X_i -\bar X)(Y_i - \bar Y)$ will have a negative and a positive factor, thus a negative product. So such points will make a negative contribution to $\sum (X_i -\bar X)(Y_i - \bar Y).$

The denominator is essentially the product of the numerators of the standard deviations of X and Y. The effect of the denominator is to make $r$ a quantity without units. In the US system of measurements the numerator has units 'foot-pounds', and the denominator has the same units, so $r$ has no units. If the subjects were weighed and measured in the metric system, the correlation of their weights and heights would be numerically the same as if they were weighed and measured in the US system.

Also, inclusion of the denominator scales correlations $r$ so that they lie between $-1$ and $+1,$ where $r = 1$ means the points perfectly fit an upward sloping line (regardless of the numerical value of the slope, which has units), and $r = -1$ means the points perfectly fit a downward sloping line.

If either the SD of the X's or the SD of the Y's is 0, then the points lie on either a vertical line or a horizontal line, respectively. In either case the denominator of $r$ would be $0$ and the correlation is not defined.

In the plot below, there is a strong linear component to the positive association of X and Y: $r = 0.968.$ The horizontal and vertical grid lines cross at $(\bar X, \bar Y).$ Each of the dark green points makes a positive contribution to the numerator of $r,$ as discussed above. Also, the two red points make (slight) negative contributions to the numerator of $r.$

enter image description here

BruceET
  • 51,500
  • why are X distances from mean multiplied by Y distances? I understand it's because you will get positive or negative correlation but if a data point lies on X or Y mean the data point coordinate on the other axis will be useless/lost (the data point will count only as a whole as a little lost in correlation) because the multiplication will give 0. Is this loss of information the best we can do? What am i missing? – fisio Mar 18 '22 at 17:23
  • The definition is what it is. – BruceET Mar 18 '22 at 17:37