Questions tagged [linear-regression]

For questions about linear regressions, an approach for modeling the relationship between a scalar dependent variable y and one or more explanatory variables.

In statistics, linear regression is an approach for modeling the relationship between a scalar dependent variable y and one or more explanatory variables (or independent variables) denoted X. The case of one explanatory variable (independent variable) is called simple linear regression. For more than one explanatory variable (independent variable), the process is called multiple linear regression.

In linear regression, the relationships are modeled using linear predictor functions whose unknown model parameters are estimated from the data. Such models are called linear models. Most commonly, the conditional mean of y given the value of X is assumed to be an affine function of X; less commonly, the median or some other quantile of the conditional distribution of y given X is expressed as a linear function of X. Like all forms of regression analysis, linear regression focuses on the conditional probability distribution of y given X, rather than on the joint probability distribution of y and X, which is the domain of multivariate analysis.

Source: https://en.wikipedia.org/wiki/Linear_regression

1306 questions
3
votes
1 answer

What's the variance of intercept estimator in multiple linear regression?

Suppose a linear regression model $Y=Xβ+ε$ where $X$ is an $n$-by-$(k+1)$ matrix and $\epsilon$ follows $N(0,\sigma^2I_n)$. $k$ is the number of explanatory variables. The first column of $X$ is one (intercept). Or we can write in this form:…
3
votes
1 answer

Obtaining Uncertainty in Linear Regression

I'm trying to find the uncertainty values for a set of data points' slope and intercept. The values are: $(0.18751,0.512332), (0.17076, 0.511825), (0.23204,0.513665), (0.20878, 0.512986), (0.17172, 0.511855), (0.19006, 0.512479)$ and the equation…
Max
  • 1,620
3
votes
1 answer

How do I find the values of $\theta_0$ and $\theta_1$ in linear regression?

Suppose I am provided with a dataset of $X$ and $Y$ values, and if I would like to use linear regression $\left( hθ(x)=\theta_0+\theta_1 x \right)$, to estimate $Y$ as a function of $X$, then how can I find values for both $\theta_0$ and $\theta_1$…
2
votes
1 answer

Deriving OLS Estimation with Measurement Error

I am trying to assess the bias of a regression model such that the true model, $y=X\beta+u$ is amended such that the true $X$ is replaced with $\widetilde{X} = X + \epsilon$, where $\epsilon$ is a vector of measurement errors. I'm working with the…
Kate
  • 21
2
votes
1 answer

Change in R-square as a result of linearly combining independent variables in linear regression

Can one improve the R-square in a linear regression by linearly combining some of the independent variables? My intuition is that the fit gets (weakly) worse because the result is a more constrained regression. Below is a specific example. (1)…
Alireza
  • 21
2
votes
1 answer

why does linear regression give a good result here?

In Excel I created some data. The variables are Time Length Width height ChocolateQuality price The response variable is Price. I used the following formula to generate the values for price: price = B2*C2*D2*5 + 2*E2 + SIN(A2/3.14) or price =…
2
votes
1 answer

Linear regression - changes in the form of Beta coefficients

We all know that we can fit a linear regression model even in cases when our predictors are not linearly related to the dependent. Like the example below (polynomial regression) - (y = a + b X^2) can be written as (y = a + b Z) where Z = X^2 My…
2
votes
2 answers

Relationship between OLS estimates of slope coefficients of simple linear regression Y on X and X on Y

Assume a model $y = \beta_0 + \beta_1x + u$. Given a sample $(x_i, y_i)_{i=1}^n$, we can find the OLS estimates of $\beta_1$, $\hat{\beta_1}$. Then suppose that we assume another model $x = \gamma_0 + \gamma_1y + \varepsilon$. Then we also can…
D F
  • 1,342
2
votes
1 answer

Why is Y = B0 + B1 * (X^2) linear and Y = B0 + (B1^2) * X not linear in linear regression?

I just started taking a regression analysis course. My professor was emphasizing that the class only focuses on Linear regression. The professor gave an example that: > (1) Y = B0 + B1 * (X^2) is linear, and > (2) Y = B0 + (B1^2) * X is NOT…
tidbits
  • 23
2
votes
1 answer

For a linear regression of $\{(i,y_i)\}_{i=0}^{n-1}$, where $(y_i)$ is increasing and non-negative, is the $y$-intercept at least $-y_{n-1}$?

Suppose we have a set of data points $\{(i,y_i)\}_{i=0}^{n-1}$, where $y_i$ are non-negative integers and where $(y_i)_{i=0}^{n-1}$ is an increasing sequence. Question: In a simple linear regression for this data set, is it true that the…
1
vote
1 answer

How to solve a linear ridge regression.

The below is some part of this thesis. https://kaiminghe.github.io/publications/pami12guidedfilter.pdf. I could not understand how equation (4) is calculated. I know that I need to do “partial differentiation $=0$” of (2). I tried to get equation…
CamFly
  • 31
1
vote
0 answers

Approximating weight of individual items from sum of their weight

Problem I have a list of orders, approximation of their total weight and list of items they contain. I need to determine approximate weight of individual items. In other words, I have a few thousand of equations like these: $$ \begin{cases} …
Draex_
  • 123
1
vote
0 answers

fundamental error and variance decompositions: bias + variance and explained/non-explained variance

As I understand there are two fundamental decompositions behind linear regression the general bias + variance decomposition total variance = explained variance + non-explained variance how do these two relate to each…
Daniel S.
  • 823
1
vote
1 answer

expand the coefficient formula of multiple linear regression with intercept

If we consider the multiple linear regression with intercept: $$y = \alpha + \beta_1x_1 + \cdots + \beta_nx_n,$$ it is the well-know formula of solution: $$\theta = (M^TM)^{-1}M^Ty.$$ Here $\theta = (\alpha, \beta^T)^T,$ $\alpha$ is intercept;…
1
vote
1 answer

Linear regression estimate $\hat{\beta}$

Follows the enunciated of the question and after my reasoning. In short, I didn't arrived to the answer of the book but I know the reasoning. Follows the question: Follows the hint and the answer of the book: My reasoning: Let A =…
1
2 3 4 5 6