1

As a student learning Applied Regression Analysis, I come from a background with very little information about this topic.

I understand that given $y = \beta_0 + \beta_1x_1 + \epsilon$

$E(y\mid x) = \beta_0 + \beta_1x_1$ is an exact linear relationship.

However, if we use a function as log($x$), sin($x$), or cos($x$), will the relationship continue to be linear?

For instance, in the parameters $\beta$,

$E(Y\mid x_1,x_2) = \beta_1 x_1 + \beta_2 \log(x_2)$, is this linear?

Clearly, $\beta_1x_1$ is linear; however, is the part $\beta_2\log(x_2)$ also linear? From a calculus point of view, we know that the logarithm function isn't exactly linear in the sense it is a polynomial of degree one.

Any hints are much appreciated.

Michael Hardy
  • 1
Julian Park
  • 205

2 Answers2

1

By linear model we shall mean a mathematical equation that involves random variables , mathematical variables and parameters and that is linear in parameters.

Your $E(Y\mid x_1,x_2) = \beta_1x_1 + \beta_2 \log x_2$ is linear. Here your parameters are $\beta_1$ and $\beta_2$ and they are linear . Also note that $x_1$ and $x_2$ if given they are fixed and consequently $x_1$ and $\log x_2$ are fixed.

NB: $y=\beta_0 + \beta_1 \exp (-x_1) + \beta_2 \log ~x_2 +e$ ,where $e\sim N(0,\sigma^2)$, is Linear Model.

However $y=\beta_0 \exp(-x_1 \beta_1)+e$ ,where $e\sim N(0,\sigma^2)$, is not linear in parameters and is not a Linear Model.

Michael Hardy
  • 1
Argha
  • 4,671
1

Linear implies "linear in the parameters." For linear regression, the most important assumption is that the residuals $\epsilon_i$ are normally-distributed $\sim \cal{N}(0, \sigma^2)$. Therefore, if your $x$-variables are skewed with right tails (log-normally-distributed), left tails, and basically not normally distributed, the residuals will not be normally distributed. Transforms on $x$ are only commonly performed in linear regression into order to obtain normally distributed residuals, which is an absolute requirement. If all of the $x$-variables used for a linear regression models are normally distributed with little skewness, then you commonly don't have to use transforms on $x$.

Please note: sometimes a regression model is presented in terms of a non-linear function for which non-linear regression is used (partial derivatives, gradient descent, etc.). However, in those cases, the problem statement is in terms of a non-linear function. When you are using linear regression, you are simply solving a system of linear equations.

Overall, when using linear regression, you must meet a number of assumptions such as:

  1. normally-distributed residuals
  2. homoscedasticity (variance of $y$ across values of $x$ must be equal)
  3. independence (each object or record must be independent of one another, i.e. zero correlation or zero covariance).
  • 1
    If normal distribution is weakened to an assumption of finite variance and expectation $0$, and independence is weakened to uncorrelatedness, and homoscedasticity is left as it is, then you have just the assumptions used in the Gauss--Markov theorem, whose conclusion is that the least-squares estimates are in a certain sense optimal. – Michael Hardy Dec 27 '13 at 17:59