Questions tagged [regression]

This tag is for questions on (linear or nonlinear) regression, which is a way of describing how one variable, the outcome, is numerically related to predictor variables. The dependent variable is also referred to as $~Y~$, dependent or response and is plotted on the vertical axis (ordinate) of a graph.

Regression is a statistical measurement used in finance, investing and other disciplines that attempts to determine the strength of the relationship between one dependent variable (usually denoted by $~Y~$) and a series of other changing variables (known as independent variables).

Types of Regression –

  • Linear regression
  • Logistic regression
  • Polynomial regression
  • Stepwise regression
  • Stepwise regression
  • Ridge regression
  • Lasso regression
  • ElasticNet regression

The two basic types of regression are linear regression and multiple linear regression.

The general form of each type of regression is:

  • Linear regression: $~Y = a + b~X + u~$
  • Multiple regression: $~Y = a + b_1~X_1 + b_2~X_2 + b_3~X_3 + ... + b_t~X_t + u~$

Where:

  • $Y =~$ the variable that you are trying to predict (dependent variable).
  • $X =~$ the variable that you are using to predict Y (independent variable).
  • $a =~$ the intercept.
  • $b =~$ the slope.
  • $u =~$ the regression residual.

There are multiple benefits of using regression analysis. They are as follows:

$1.~$ It indicates the significant relationships between dependent variable and independent variable.

$2.~$ It indicates the strength of impact of multiple independent variables on a dependent variable.

Reference:

https://en.wikipedia.org/wiki/Regression_analysis

This tag often goes along with the tag.

2700 questions
1
vote
1 answer

How to gather useful information from a residue plot

You can usually see how good your linear regression line is by looking at the residue plot. If you see the points randomly distributed, you're good. But if you see a pattern, it means there is something wrong with your model; you perhaps need a…
Ylyk Coitus
  • 970
  • 2
  • 10
  • 18
1
vote
1 answer

How to interpret these regression values?

If GPA(gpa after fall semester in college) is the dependent variable and SAT (score on the SAT) is the independent variable and I have the following parameter estimates: Intercept: .66306 SAT: .00193 Than would I assume that every additional point…
Gamecocks99
  • 1,023
1
vote
2 answers

Linear regression question

I don't understand the following derivation: $$ e_i = y_i - ax_i - b$$ $$ e_i = (y_i - \bar{y}) - a(x_i - \bar{x}) - (b - \bar{y} + a \bar{x}) $$ I don't really understand what they do and why they do it. To clarify: $e_i = y_i - \hat{y}_i$, where…
Ylyk Coitus
  • 970
  • 2
  • 10
  • 18
1
vote
1 answer

Forecasting using multiple regression

I have data in the form given below, and I want to perform forecasting using multiple regression. I found definition of multiple regression from this link: http://otexts.com/fpp/5/1/ . I have these questions: (1) Can we forecast value of Y (the…
1
vote
1 answer

Computing SSREG and SSRES (Regression)

SSREG is calculated with $\sum_{i=1}^{n} (\hat Y_i - \bar{Y})^2$ which is the regression sum of squares SSRES is calculated with $\sum_{i=1}^{n} ( Y_i - \hat Y_i)^2$ which is the residual sum of squares If we are given $$\sum_{i=1}^{n} X_i = 323.02,…
Tree Garen
  • 709
  • 5
  • 15
1
vote
1 answer

estimator lasso

I would like to know if you could either give me a hint to answering the following problem or a reference (article, book, etc) to have a better idea on how to address it the prove for LASSO estimator is the solution to the optimization problem…
1
vote
2 answers

In a normal linear model (with intercept), show that if the residuals satisfy $e_i = a + \beta x_i$, then each residual is equal to zero.

In a normal linear model (with intercept), show that if the residuals satisfy $e_i = a + \beta x_i$, for $i = 1\dots n$, where $x$ is a predictor in the model, then each residual is equal to zero. I'm not really sure how to do this, I tried writing…
mrnovice
  • 5,773
1
vote
2 answers

Non-linear regression - least square regression

I was trying to get some insight into how to solve non-linear regression model problems. Unfortunately, I've never attended a lecture on statistical math. Here is the link: In page number 4, they said, calculate the least square regression. I don't…
1
vote
1 answer

How to find autoregressive coefficients of ARMA model

I am searching various sources to find the method of manually calculating coefficients of Auto Regressive Moving Average model. The following is the text I found in a book. My question is how a1 and a2 are found. Can any one please help.…
1
vote
2 answers

Question about logistic regression

A logistic regression is meant for a binary/categorical variable. Sort of like age vs baldness. 1) So, does the "S-curve" regression equation output give the odds of having that condition for a given x-value (eg: age), since the values go from 0…
JackOfAll
  • 4,701
1
vote
0 answers

Real World Examples of Sinusoidal and Logistic Functions (and Regression)?

Anyone ever use a real world example (or data) of either: 1) Sinusoidal Regression 2) Logistic/Log Regression This can alternatively be answered by identifying any real world functions that are: 1) Sinusoidal 2) Logistic/Log Regression …
JackOfAll
  • 4,701
1
vote
1 answer

What is the partial derivative of the loss function (root mean squared) w.r.t. theta when theta is the exponent of x?

I'm trying to solve a regression problem using python 3 without an machine learning libraries. The input data consists of a csv file of x,y floats which should fit the hypothesis: y = x^theta I need to use regression to find the value of…
Alex Ryan
  • 293
1
vote
1 answer

Linear regression: b1 has the minimum variance among all unbiased linear estimators of beta1

There is a proof provided in Applied Linear Regression Models (1983) by Kutner et al. (Page 64), which is quite clear and easy to understand, except one point, namely, it assumes that $\sum k_i d_i = 0$, from the "restrictions on $k_i$ and $c_i$",…
qed
  • 1,083
1
vote
1 answer

Find X in a 4PL Curve Regression

I'm working on a project where I need to replicate a calculation that is currently done by a legacy system, let's call this legacy system of 4P. In the company nobody knows how the calculation is done, my goal is to find out how it works. Let's…
1
vote
1 answer

OLS Estimator Summation Proof

In deriving the OLS estimators for 1 variable, I am trying to convert $$ \begin{align} \hat{\beta_1} = \frac{\sum_{i=1}^N X_iY_i- N\bar{X}\bar{Y} }{\sum_{i=1}^NX^2_i - N(\bar{X})^2} \end{align} $$ to the final form $$ \hat{\beta_1} =…
bryan.blackbee
  • 4,161
  • 2
  • 29
  • 50