2

Let $X=(1,2,3,...,20)$. Suppose that $Y=(y_1,y_2,...,y_{20})$ with $y_i=x_i^2$ and $Z=(z_1,z_2,...,z_{20})$ with $z_i=e^{x_i}$. Pearson correlation coefficient is defined by formula \begin{equation} \rho(X,Y)=\frac{\sum_{i=1}^{20} (x_i-\bar{x})(y_i-\bar{y})}{\sqrt{(\sum_{i=1}^{20}(x_i-\bar{x})^{2})(\sum_{i=1}^{20}(y_i-\bar{y})^{2})}} \end{equation}

If $\rho(X,Y)=1$, we can say that $X$ and $Y$ have a linear correlation. If $0.7\leq\rho(X,Y)<1$ then $X$ and $Y$ has a strong linear correlation, if $0.5\leq\rho(X,Y)<0.7$ then $X$ and $Y$ has a modest linear correlation, and if $0\leq\rho(X,Y)<0.5$ then $X$ and $Y$ has a weak linear correlation. Using this formula, we get $\rho(X,Y)=0.9$ and $\rho(X,Z)=0.5$. However, the relationship between $X$ and $Y$ is actually quadratic but they have the high correlation coefficient that indicate linear correlation.

So, my question is what is "linear correlation" actually between $X$ and $Y$ ? Since $\rho(X,Z)=0.5$ indicate the modest correlation coefficient, what is another intepretation of this value? What is the difference between $\rho(X,Y)$ and $ \rho(X,Z)$, noting that $Y$ and $Z$ is not a linear function of $X$.

beginner
  • 2,117
  • 4
  • 27
  • 50

1 Answers1

2

A linear function is given as $ax+b$ for some coefficients $a$ and $b$. If you have $y=ax+b$, then you can obtain $y$ from $x$ via some mean shifting and scaling, which are quite simple operations. If you have two such functions, then whenever you compute $a$ and $b$, you are done. They are completely correlated. If there is another function and you do linear regression and you found some $a$ and $b$ but the variance of the error of this regression is quite large then your correlation will degrade, it will seem that you need quite much than just mean shifting and scaling.

In your example $\exp(x)$ is a quite non-linear function especially compared to $x^2$. You can see this via Tailor expansion; $\exp(x)$ has infintely many terms, indicating the strength of the non-linearity compared to $x^2$ which is a single term. Therefore the correlation is much lesss.

Seyhmus Güngören
  • 7,848
  • I also try $X=(1, 1.1 ,1.2,...,5)$ and $Z=(z_i)$ with $z_i=e^{x_i}$ and we get $\rho(X,Z)=0.89$. How come? – beginner Mar 18 '13 at 00:17
  • do you have a programming software? plot them. You will see immediately why. – Seyhmus Güngören Mar 18 '13 at 00:25
  • SO your argument using taylor(MacLaurin) expansion is not true in general right? – beginner Mar 18 '13 at 00:30
  • it is true in general for the same input data. Try to plot everything we've talked about. You will understand what i am talking about. – Seyhmus Güngören Mar 18 '13 at 00:36
  • @ Seyhmus Güngören : WHat is the mathematical ecplaination about this phenomena? – beginner Mar 18 '13 at 00:37
  • It is related to how much you are far from being linear. If you are closer then you will get a higher coefficient, else lower. It might happen that two functions can behave different for given input data. For example one function can be quite linear near $0$ and the other can be quite linear around $3$. If your input is around $0$ then the first function and if your input is around $3$ then the second function will give you higher coefficient. – Seyhmus Güngören Mar 18 '13 at 00:40
  • @Sehmus: But the argument is based on graphic, not the mathematical argument? – beginner Mar 18 '13 at 00:58
  • No it is not based on a graphic. One can insert the related functions into the formula and show that one is greater than the other. I am telling you the general idea, which many formulas will not give. – Seyhmus Güngören Mar 18 '13 at 12:51