Questions tagged [linear-regression]

For questions about linear regressions, an approach for modeling the relationship between a scalar dependent variable y and one or more explanatory variables.

In statistics, linear regression is an approach for modeling the relationship between a scalar dependent variable y and one or more explanatory variables (or independent variables) denoted X. The case of one explanatory variable (independent variable) is called simple linear regression. For more than one explanatory variable (independent variable), the process is called multiple linear regression.

In linear regression, the relationships are modeled using linear predictor functions whose unknown model parameters are estimated from the data. Such models are called linear models. Most commonly, the conditional mean of y given the value of X is assumed to be an affine function of X; less commonly, the median or some other quantile of the conditional distribution of y given X is expressed as a linear function of X. Like all forms of regression analysis, linear regression focuses on the conditional probability distribution of y given X, rather than on the joint probability distribution of y and X, which is the domain of multivariate analysis.

Source: https://en.wikipedia.org/wiki/Linear_regression

1306 questions
1
vote
0 answers

Effect of multicollinearity on the regression coefficients of a linear regression

I recently received an exam question where I was asked to compare the regression coefficients of 2 linear regression models (see below) as well as their predicted values. The dependent variables var1 and var2 are highly correlated with each other. y…
1
vote
1 answer

Geometric Interpretation of Linear Regression

I was reading a tutorial written on Linear Regression by Avi Kak (https://engineering.purdue.edu/kak/Tutorials/RegressionTree.pdf). There is a part about geometric interpretation of linear regression on pg.19. The optimum solution for β~ that…
1
vote
0 answers

Covariance between estimated y and observed y(in simple linear regression)

I wonder what is $Cov(y,\hat{y})$ the Covariance between estimated $\hat{y}$ and observed value $y$, in the situation of simple linear regression. Some Youtube tutor said it's 'O', but i can't agree with that. You know, the linear regression model…
서영빈
  • 137
  • 1
  • 8
1
vote
0 answers

Understanding Estimation of Liner Co-efficients

As I was going through some documentation on how to estimate the linear co-efficients (intercept and weights) here in this page (https://ml-cheatsheet.readthedocs.io/en/latest/linear_regression.html) I could not understand the following…
joesan
  • 111
1
vote
1 answer

Why does feature scaling by a constant vs. by a function produce different results in linear regression?

If I have some data points $(x_1, x_2,\dots,x_n,y)$ where $y$ is the dependent variable and want to perform linear regression. If I perform regression with $x_1, x_2,\dots,x_n$ and $y$ vs. scaling one or multiple $x_i$ feature by some constant (say…
Yandle
  • 885
1
vote
0 answers

Converting back to unnormalized domain after normalizing

I'm trying to perform a linear regression on a dataset however, without normalizing the data I couldn't achieve any results therefore I was forced to normalize it. Now my graph looks fine for the normalized domain where the x and y axis are z scores…
Craig
  • 652
1
vote
0 answers

The minimal possible value of total sum of squares for linear regression

I have a regression model $y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 +u$ It is known that sample means of both $x_1$ and $x_2$ are zero, moreover the error term is said to be homoskedastic, the standard error of regression and standard errors of OLS…
D F
  • 1,342
1
vote
0 answers

Elliptical Confidence Set Calculation

I got stuck in a homework question: In Linear regression model with assumption $\varepsilon_{i} \sim \cal{N}(0, \sigma^{2})$, iid. $$Y_{i} = X_{i}^{\intercal}\theta^{*} + \varepsilon_{i}, ~ i = 1, \cdots, n, ~ \mathbb{E}\varepsilon_{i} \sim…
adosdeci
  • 165
1
vote
1 answer

Linear regression coefficient

I'm studying linear regression with Draper's Applying regression analysis book There is a formula for coefficient b1 which is clear to me: $$ b_{1}= \frac{n\sum x_{i}y_{i}-\sum x_{i}\sum y_{i}}{n\sum x_{i}^2 - \sum x_{i}\sum x_{i}} $$ What I can't…
amplifier
  • 205
1
vote
0 answers

Linear regression: residual values do not have zero mean

I performed a linear regression model y~x in R by running lr <- lm (y~x, data = train) p <- predict (lr, newdata = test) error <- p - test$y It seems that error does not have zero mean (in fact, mean(error) = -5), it is right-skewed and I could not…
1
vote
2 answers

Calculating Slope of Line of Best Fit with Y-intercept of 0

I have a TI-83 Plus CE that I can use to calculate the line of best fit for a set of points. However, I cannot force that line to go through (0,0). I was wondering how I should find the line of best fit that goes through (0,0) by hand or with a…
WIR3D
  • 125
1
vote
1 answer

F-Test - linear regression

I have to solve the following problem: After data cleansing and using ordinary least squares we derive the following formula: $\hat{Y}_i=3,85+0,78X_i$ where $Y_i$ is the price of a stock of the i-th company (in \$) and $X_i$ the percentage that the…
1
vote
1 answer

How to make a sigmoid between $-1$ and $1$

I am trying to make a sigmoid which ranges from $-1$ to $1$. I have found the formula $\dfrac{1}{​1+​e^{​\, ‑x}}-​0.5$ which creates a sigmoid between $-0.5$ and $0.5$ I need an equation that I can input $x$ and will get you the value for $y$ E.g.…
Rlz
  • 227
1
vote
0 answers

Linear or Intrinsically Linear Regression Model?

I have the following model and I'm not sure whether its linear or if it can be made linear. I don't know if it can be considered linear already with the log and X^2 or if I have to rewrite it. Model
1
vote
1 answer

Curve Fitting: Multidimensional input

According to Bishop's book, if your input is a multidimensional vector $x$, then non kernel based basis function models suffer from the curse of dimensionality: But the above picture is based on polynomial basis functions in one dimension,…