Questions tagged [regression]

This tag is for questions on (linear or nonlinear) regression, which is a way of describing how one variable, the outcome, is numerically related to predictor variables. The dependent variable is also referred to as $~Y~$, dependent or response and is plotted on the vertical axis (ordinate) of a graph.

Regression is a statistical measurement used in finance, investing and other disciplines that attempts to determine the strength of the relationship between one dependent variable (usually denoted by $~Y~$) and a series of other changing variables (known as independent variables).

Types of Regression –

  • Linear regression
  • Logistic regression
  • Polynomial regression
  • Stepwise regression
  • Stepwise regression
  • Ridge regression
  • Lasso regression
  • ElasticNet regression

The two basic types of regression are linear regression and multiple linear regression.

The general form of each type of regression is:

  • Linear regression: $~Y = a + b~X + u~$
  • Multiple regression: $~Y = a + b_1~X_1 + b_2~X_2 + b_3~X_3 + ... + b_t~X_t + u~$

Where:

  • $Y =~$ the variable that you are trying to predict (dependent variable).
  • $X =~$ the variable that you are using to predict Y (independent variable).
  • $a =~$ the intercept.
  • $b =~$ the slope.
  • $u =~$ the regression residual.

There are multiple benefits of using regression analysis. They are as follows:

$1.~$ It indicates the significant relationships between dependent variable and independent variable.

$2.~$ It indicates the strength of impact of multiple independent variables on a dependent variable.

Reference:

https://en.wikipedia.org/wiki/Regression_analysis

This tag often goes along with the tag.

2700 questions
2
votes
4 answers

Seeking a function based on its level set

I'm trying to create a function for a research project, but I fear my math knowledge is insufficient to derive it from the attached diagram I've created showing its desired behavior. I'm hoping someone will recognize the behavior and provide some…
Jeff
  • 33
2
votes
1 answer

Multiple vs Single Linear Regression

I'm having trouble understanding the relationship between multiple and single linear regression. I have six variables $(x_1, \dots, x_6)$ I'm using in my model. If I check each one individually against $y$, all but $x_2$ come up significant (and…
1
vote
0 answers

Regression on even function?

Is there any test for whether or not the regression function is even? Suppose we have a model: $Y=g(X, \epsilon)$, where $Y, X$ are both one dimensional. My questions is how do we test for $g$ is an even function or not? For example…
bankrip
  • 566
1
vote
1 answer

what is the difference between 'estimate of residual standard error' and 'residual standard error'?

What is the difference between 'estimate of residual standard error' and 'residual standard error'? Can someone please provide the formulas? Thanks!
mike
  • 53
1
vote
0 answers

Round robin logistic regression

I have a poll with four answers (A,B,C,D) and response information about people who have taken that poll. I have created four models (one for each of the answers) in a one vs all. i.e. the model for answer A has answer A as a 1 and the rest of the…
sedavidw
  • 610
1
vote
1 answer

Regression Model with even function?

Is there any method to test if the mean function, $f(x)$, of a regression model $y=f(x)+\epsilon$ is even or not?
bankrip
  • 566
1
vote
1 answer

Modeling non-linear data using least squares best fit

I have some data for liquid viscosity as a function of pressure and temperature. I would like to learn how come up with a single equation that would determine this fluid's viscosity with pressure and temperature as inputs. How can I go about doing…
Armadillo
  • 525
1
vote
0 answers

Good MSE doesnt imply good prediction in logistic regression?

I am writing some code for regularized logistic regression. I observe this interesting phenomena and wonder if it is a normal thing or just my code is wrong. For loss function, I am using the logistic loss function ( maximize likelihood of binary…
1
vote
1 answer

Meaning of $\mathcal{I}_t$ in assumption $E[u_t\mid\mathcal{I}_t]$ of distributed-lag model

When considering \begin{equation*} y_t = \beta_0 + \beta_1 x_t + \ldots + \beta_r x_{t-r} + u_t \end{equation*} an assumption made is \begin{equation*} E[u_t\mid\mathcal{I}_t] = 0 \end{equation*} What exactly does $\mathcal{I}_t$ stand for? I know…
1
vote
0 answers

Problem on Linear Regression

Consider the following 2-variable linear regression where the error $e_i$ 's are independently and identically distributed with mean 0 and variance 1; $y_i = α + β(x_i − \bar x) + e_i , i = 1,2,...,n.$ Let $ \hatα$ and $ \hat β$ be ordinary least…
1
vote
1 answer

Linear regression or ANOVA with unordered independent variable

I have a set of data, let's say describing a group of people. Let's say we know their income and color of hair: N | hair | income ---|--------|------ 1 | brown | £2000 2 | black | £1400 3 | brown | £1800 4 | red | £1600 5 | brown |…
Pavel S.
  • 111
1
vote
2 answers

Is it possible to fit any regression line to a set of data points?

If you have a set of data points (x1,y1), (x2,y2),...,(xn,yn) And you know it fits a trend y=f(x) where f(x) is a known function, Say for example: f(x)=A*x*sin(B*x) Providing f(x) is known, is it possible to fit a regression line to this data and…
Edd
  • 147
1
vote
0 answers

Predicting y from a log-linear regression

I was wondering if someone could explain to me the very last step on the right hand slide. Why is do we have a sum rather than a product. Thank you very much.
1
vote
1 answer

help with using the "simple regression (least squares) method" of forecasting

This problem is from an engineering management textbook (Morse & Babcock, 5th ed) : 2005 $48k 2006 $64k 2007 $67k 2008 $83k "What is the sales forecast for 2009, using the simple regression (least squares) method?" The book works…
1
vote
0 answers

Linear Regression with multiple equations

I am trying to implement a linear regression algorithm to fit a set of "true" points with their "observed" location. The points are specified using spherical coordinates on a unit sphere. I have a model that relates true to observed as…
OzBandit
  • 111