0

Consider the following LASSO problem $min_{\beta} \sum\limits_{i=1}^n(y_{i}-\sum\limits_{j=1}^p x_{ij}\beta_{j})^2$ , subject to $\sum\limits_{j=1}^p|\beta_{j}|\leq t$

where $t \geq 0$ is a constant.

(a) If $t = 0$, compute $\hat{\beta}_{j}^{lasso}$ , for $j = 1, . . . , p$.

(b) Define $t_{0}= \sum\limits_{j=1}^p \hat |{\beta}_{j}^{ols}| $. Prove that, if $t \geq t_{0}$, then

$\hat{\beta}_{j}^{lasso} $=$ \hat{\beta}_{j}^{ols}$

My approach: for (b) $ \hat{\beta}_{j}^{ols}=\sum\limits_{i=1}^n(y_{i}-\hat{y})^2$ and $\hat{\beta}_{j}^{lasso} = \sum\limits_{i=1}^n(y_{i}-\hat{y})^2 + \lambda \sum\limits_{j=1}^p |\beta_{j}|$,

when $\lambda =0 $ we can get get $\hat{\beta}_{j}^{lasso} $=$ \hat{\beta}_{j}^{ols}$

I am not sure about the approach, also don't know how to do part (a). Please guide.

Proababilty
  • 141

1 Answers1

1

Are you a student at UA? Let me give you some hint.

Consider the problem below.

Find real number x and $y$ which minimize $x^2 + y^2$ subject to $|x|+|y|\le t$.

(a)If t = 0, find the x and $y$.

(b)If t = 5, find x and $y$.

For part (a), the restriction is $|x|+|y|\le 0$, so x must be 0 and $y$ must be 0. Otherwise $|x|+|y|> 0$.

For part (b), the global solution for minimizing $x^2 + y^2$ without any restriction is x=0, $y$=0. This global solution without any restriction satisfied the restriction $|x|+|y|\le 5$. Thus, the solution for minimize $x^2 + y^2$ subject to $|x|+|y|\le 5$ is also x=0, $y$=0.

Please work on your question with the hint.

Mizzle
  • 165