Questions tagged [linear-regression]

For questions about linear regressions, an approach for modeling the relationship between a scalar dependent variable y and one or more explanatory variables.

In statistics, linear regression is an approach for modeling the relationship between a scalar dependent variable y and one or more explanatory variables (or independent variables) denoted X. The case of one explanatory variable (independent variable) is called simple linear regression. For more than one explanatory variable (independent variable), the process is called multiple linear regression.

In linear regression, the relationships are modeled using linear predictor functions whose unknown model parameters are estimated from the data. Such models are called linear models. Most commonly, the conditional mean of y given the value of X is assumed to be an affine function of X; less commonly, the median or some other quantile of the conditional distribution of y given X is expressed as a linear function of X. Like all forms of regression analysis, linear regression focuses on the conditional probability distribution of y given X, rather than on the joint probability distribution of y and X, which is the domain of multivariate analysis.

Source: https://en.wikipedia.org/wiki/Linear_regression

1306 questions
1
vote
1 answer

Fisher F test - linear regression

We consider $Y=X\beta+\epsilon$, $\epsilon \sim N(0,\sigma^2 I)$. Let H$_0$ is $\beta_0=\dots\beta_k=0$ vs exist $i\in{0,1,\dots,k}: \beta_i\neq 0$ How to proof that if H$_0$ is true then $F=\frac{n-1-k}{k}\frac{\sum_{i=1}^n(\hat{Y}_i-\overline…
AnnaGlo
  • 183
1
vote
1 answer

The unbiased estimator of the variance of $\widehat{\beta}_1$ in simple linear regression

Given the simple linear regression model $Y = \beta_0 + \beta_1x + U$, where $U\sim N(0,\sigma^2)$, I know how to derive \begin{equation} \text{Var}[\widehat{\beta}_1] = \dfrac{\sigma^2}{\sum_{i=1}^n(x_i - \overline{x})^2} \end{equation} However, I…
Matti
  • 111
1
vote
0 answers

How to calculate the variance of regression prediction bias

In linear regression, if we know $MSE$, which is $MSE=\frac{1}{n}\sum_{i=1}^{n}(\hat{y_i}-y_i)^2$, can we know $Var(|\hat{y_i}-y_i|)$? The reason I am trying to find the variance of the bias is that I want to know the error level for the individual…
daydayup
  • 435
1
vote
0 answers

For a linear regression $f$ of $\{(i,y_i)\}_{i=0}^{n-1}$, where $(y_i)$ is non-negative and increasing, is $|y_i - f(i)|$ at most $y_{n-1}$?

Suppose we have a set of data points $\{(i,y_i)\}_{i=0}^{n-1}$, where $y_i$ are non-negative integers and $(y_i)_{i=0}^{n-1}$ is an increasing sequence. Question: In a simple linear regression $f$ for this data set, is it true that the absolute…
0
votes
0 answers

Linear regression in ML

If my model is perfect then that means $SSE = 0$, which means $SST = SSR$ as $SST = SSR + SSE$, which means $R^2 = 1 - 1 = 0$, as $R^2 = 1 - SSR/SST$, but $R^2$ should have given me $1$ and not $0$, I don't know what I am not understanding.
KERIS
  • 1
0
votes
0 answers

Conditional Mean simulation

The joint p.d.f of $f(x,y)=2 $ if $0
Upstart
  • 2,613
0
votes
0 answers

Vector Isotonic Regression

I’m looking to a vector isotonic regressions. I have a set of multivariate regressions. For each regression, I want all the coefficients to be greater than or equal to the last regression. In the univariate case, this is the normal isotonic…
0
votes
0 answers

Linear Regression, what's the difference between $x$ and $x - \bar{x}$?

In Linear Regression, what's the difference between $\displaystyle\hat{\beta} = \frac{\sum_{i = 1}^{n}\left(x_i - \bar{x}\right)\left(y_i - \bar{y}\right) } {\sum_{i = 1}^{n}\left(x_i - \bar{x}\right)^2}$ and $\displaystyle\hat{\beta} =…
Caj
  • 55
0
votes
0 answers

Confidence interval of slope parameter when the intercept in known in simple linear regression model

It is one of the exercise in "Introduction to Multiple Linear Regression". Consider the usual linear regression model \begin{equation} y = \beta_0+ \beta_1 x +\epsilon \end{equation} where $\beta_0$ is known. It can be calculated that the length of…
0
votes
2 answers

Differentiate matrix expression (linear regression)

$$\frac{d}{dw} [w^TX^TXw - 2w^TX^Ty+y^Ty] = 2(X^TXw-X^Ty)$$ I do not understand how the RHS was obtained -- are there certain matrix differentiation properties which can be used to show this? Why does differentiating w.r.t. $w$ get rid of the $w^T$…
0
votes
0 answers

Finding the units of measure of the slope in this regression problem

I am not sure if I am doing this problem correctly. I have to use simple linear regression to either prove or disprove the Big Bang Theory (Y = TX), using the data, I fit a regression line without an intercept term and found a 95% confidence…
0
votes
1 answer

Expression of hat matrix diagonals (leverage).

Let X is nxp (design) matrix and $H=X({X^\top}X)^{-1}{X^\top}$. We define $h_{ij}$ is an i-th row and j-th column entry of H and $x_i^\top$ is i-th row of $X$ , then show $$h_{ii}={x_i^\top}({X^\top}X)^{-1}x_i$$ and…
0
votes
1 answer

Show $var(\beta_1)$ is minimum when $x_i$s are evenly distributed at the boundaries for linear regression

I encounter the following question Suppose that $n$ is even and the $n$ values of $x_i$ can be selected anywhere in the interval from $a$ to $b$. Show that $var(\beta_1)$ is a minimum if $n/2$ values of $x_i$ are equal to $a$ and $n/2$ values are…
Lei Hao
  • 685
0
votes
1 answer

Best Fit Line Slope

I am given three pairs of (pressure $p$, molar volume $V_m$) measurements at constant known temperature $T$: (0.750000, 29.8649), (0.500000, 44.8090), (0.250000, 89.6384). The goal is to determine the ideal gas constant $R$ plotting $p$ vs $1/V_m$…
Yerbolat
  • 343
0
votes
0 answers

Can I find a nonlinear space in which hyperbola looks linear?

In kernelized linear regression, we transform the covariate coordinates (x) such that the predictor (y) becomes a linear function of the covariates. I wonder if I can find such a transformation that does this for 1/x (which is discontinuous) or any…