derivation of regression coefficients

Question

I would like to derive the confidence interval for OLS regression but having difficulty in understanding the coefficients itself. Let me state this way, for $Y=aX+b+\epsilon$ where $X, Y, \epsilon$ are random variables with $\epsilon$ zero-mean Gaussian random variable, I can find $a, b$ by minimizing $f(a,b)=E[(Y-aX-b)^2]$ wrt $a, b$ such that $\frac{\partial f}{\partial a}=\frac{\partial f}{\partial b}=0$. However, in this derivation I am implicitly assuming that $a, b$ are constants. My problem starts here:

How can I derive confidence intervals for a constant variable, e.g. for $a,b$?

If $a,b$ are not constants but rather random variables then, my derivation fails from the beginning since in this case $E[(Y-aX-b)^2]$ involves terms like $E[bY], E[aXY]$ which I believe cannot be separated since the variables are not independent of each other.

Can you please clarify this? What are the random variables in simple OLS regression? If the coefficients are not random but rather constant then how it is possible to compute the confidence interval, since as far as I know just the random variables have confidence intervals?

Regards,

Answer 1

Confidence intervals computed mainly (or even solely) for estimators rather than for just random variables. But you are right as it depends on the sample distribution of these estimators, namely the confidence interval is derived from the fact the point estimator is a random realization of (mostly) infinitely many possible values that it can take.

Specifically, regarding the problem of regression. You should distinguish between population regression and sample regression. If you are talking about the population, i.e, $$ Y = \beta_0 + \beta_1 X + \epsilon , $$
then $\beta_0 = \mathbb{E}Y - \beta_1\mathbb{E}X$ and $\beta_1 = \frac{\text{cov(X,Y)}}{\text{var}(X)}$ are constants that minimize the MSE and no confidence intervals are needed. However, given the fact that you are dealing with some random sample $\{(Y_i, X_i)\}_{i=1}^n$ of realizations of $X$ and $Y$, you can only construct a point estimators of these constants, namely $$ \hat{\beta}_0 = \bar{Y} - \hat{\beta}_1 \bar{X}\, , \quad \hat{\beta}_1 = \frac{\sum (X_i - \bar{X}) (Y_i - \bar{Y})}{\sum (X_i - \bar{X})^2}. $$ Hence, it is necessarily to talk about CI as these point estimators will miss the true values with probability $1$.