I am studyng about the $R^2$ coeficient in a OLS regression. I would like to understand the following statement: One measure of the variability of the dependent variable $y$ is the sum of squares:
$$y'y = \langle y,y \rangle = \sum_i^n y_i^{2}$$
I as a mathematician see this as a simple inner product and I can not understand the intuition of how this can represent the variability of y. Some help??
Perhaps it’s easier to build intuition by thinking of this in a contrapositive sense—which is that any random variable with variance zero isn’t actually a random variable at all. (I’ll also mention that the proper formula for the variance includes a term that subtracts the square of the expected value $(\sum y)^2$.)
You can show this easily by noting that the variance is $Var(X) = E(X^2) - E(X)^2$. For this to be true, $X = E(X)$ everywhere, forcing it to be a constant.
The fact that the variance can be represented as the inner product of a random variable with itself (after adjusting the expected value) is a consequence of the fact that mean-zero finite-variance random variables form a Hilbert space with the traditional workings, including an inner product representing the variance/covariance and norm representing the standard deviation.
