Questions tagged [regression]

This tag is for questions on (linear or nonlinear) regression, which is a way of describing how one variable, the outcome, is numerically related to predictor variables. The dependent variable is also referred to as $~Y~$, dependent or response and is plotted on the vertical axis (ordinate) of a graph.

Regression is a statistical measurement used in finance, investing and other disciplines that attempts to determine the strength of the relationship between one dependent variable (usually denoted by $~Y~$) and a series of other changing variables (known as independent variables).

Types of Regression –

  • Linear regression
  • Logistic regression
  • Polynomial regression
  • Stepwise regression
  • Stepwise regression
  • Ridge regression
  • Lasso regression
  • ElasticNet regression

The two basic types of regression are linear regression and multiple linear regression.

The general form of each type of regression is:

  • Linear regression: $~Y = a + b~X + u~$
  • Multiple regression: $~Y = a + b_1~X_1 + b_2~X_2 + b_3~X_3 + ... + b_t~X_t + u~$

Where:

  • $Y =~$ the variable that you are trying to predict (dependent variable).
  • $X =~$ the variable that you are using to predict Y (independent variable).
  • $a =~$ the intercept.
  • $b =~$ the slope.
  • $u =~$ the regression residual.

There are multiple benefits of using regression analysis. They are as follows:

$1.~$ It indicates the significant relationships between dependent variable and independent variable.

$2.~$ It indicates the strength of impact of multiple independent variables on a dependent variable.

Reference:

https://en.wikipedia.org/wiki/Regression_analysis

This tag often goes along with the tag.

2700 questions
2
votes
1 answer

Logistic Regression with a Time Factor for Forecasting

I have a general question with regards to modelling the chance that domestic flights within the U.S. are cancelled or not using logistic regression (which I'm relatively new to). I have used 'R' to fit a logistic regression model to a vast data set…
Delvesy
  • 749
2
votes
0 answers

Equation for the Human Spine

I have data that I believe might fit some kind of semi-sinusoidal trend line - I'm trying to derive an equation for the length of individual vertebrae in various species (I am not a mathematician). I'd like to put that trend line in Excel - it is…
Henry
  • 21
2
votes
1 answer

Statistical limitations of conducting nonlinear regression of $x$ vs.$ f(y)$ as opposed to $y$ vs. $f(x)$

I derived a solution to a physics problem using a method of analysis that gives me an inversed relation, the independent variable $t$ expressed explicitly in terms of a nontrivial algebraic set of functions with the dependent variable $v$, namely …
2
votes
3 answers

Best fitted rectangle on a set of points

Given a set of points, is there a method to find a rectangle which is the best approximation of the points? For example, in this picture the blue rectangle is the best fitted rectangle.
shonku
  • 41
2
votes
0 answers

Linear regression where the error is modified

I have a set of coordinates $\{(x_1,y_1),(x_2,y_2),\ldots,(x_n,y_n)\}$ , where for every $i
NODO55
  • 146
2
votes
0 answers

Multiple vs simple linear regression

some problem which intuitively seems to work, but where I have some problems to prove. Consider two regression lines fitted by OLS, namely $\hat{y_i}=b_0 +b_1x_{i1} + b_2x_{i2}$ (1) and $\tilde{y_i}=\tilde{b_0} +\tilde{b_1}x_{i1}$ (2) where…
2
votes
2 answers

The best fit for variables in a number of equations?

Let's say I have 2 variables $x$ and $y$ and 4 equations. The parameters in capital are known parameters. $$I_1=xA_1+yB_1$$ $$I_2=xA_2+yB_2$$ $$I_3=xA_3+yB_3$$ $$I_4=xA_4+yB_4$$ What's the strategy to find the best fitting values for $x$ and $y$?…
Ruts
  • 651
2
votes
0 answers

Derivation of standard error of beta in simple linear regression

Countless web pages show the equation for the standard error of the slope in a simple linear regression. For example: SE = sb1 = sqrt [ Σ(yi - ŷi)2 / (n - 2) ] / sqrt [ Σ(xi - x)2 ] However, I have not been able to find a single web page that…
stasi
  • 21
2
votes
2 answers

Power Regression $y=Ax^B+C$

I have to do reproduce a power regression but I don't have any experience in procedures like that. I read a little bit about power fit/power regression and that a formula like $y = ax^b$ is used for this. But in the formula which was used in this…
Burak
  • 153
2
votes
1 answer

variance of multiple regression coefficients

If I consider universal kriging (or multiple spatial regression) in matrix form as: ${\bf{V = XA + R }}$ where $\bf{R}$ is the residual and $\bf{A}$ are the trend coefficients, then the estimate of ${\bf{\hat A}}$ is: ${\bf{\hat…
2
votes
0 answers

Ridge Regression Centering Proof

This is a ridge regression problem. The following two problems are equivalent: $(w_t, b_\lambda ) = argmin_{w,b}\{\sum_{i=1}^m (y_i-b-w^Tx_i)^2+\lambda w^Tw\} $ $(w_t, b_\lambda )= argmin_{w,b}\{\sum_{i=1}^m (y_i-b-w^T(x_i-\bar x))^2+\lambda w^Tw\}…
Ian
  • 45
2
votes
2 answers

vertical vs. horizontal regression

A horizontal regression is defined as the following: $$m=\frac{\sum_{i=1}^n (x_i-\operatorname{average(x)})(y_i-\operatorname{average(y))}}{\sum_{i=1}^n (x_i-\operatorname{average(x)})^2}$$ whereas a vertical regression is defined…
libjup
  • 313
2
votes
1 answer

Linear Regression Model, linearity in parameters/ variables

I am confusing with the wording here. I was reading a book on linear regression. "The primary concern for linear models is that they display linearity in the parameters. Therefore, when we refer to a linear regression model we generally assume that…
David
  • 176
2
votes
1 answer

LMS Update Rule

I'm starting to study machine learning using Andrew Ng's class notes. I understand conceptually how linear regression works, but am having trouble with this equation: $$ \theta_j := \theta_j + \alpha\left(y^{(i)} - h_\theta (x^{(i)})\right)…
Ben g.
  • 21
2
votes
0 answers

combining multiple regression outputs

Suppose I have multiple regressions, along with their r-squares, standard-errors, etc.: $y(t) = \alpha_1 + \beta_1 x(t) + e_1$, where $t \in (\tau_0, \tau_1)$ $y(t) = \alpha_2 + \beta_2 x(t) + e_2$, where $t \in (\tau_0, \tau_2)$ $...$ $y(t) =…
uday
  • 300
1 2
3
15 16