1

I try to figure out a proof of Keith Knight and Wenjiang Fu regarding the asymptotic property of Lasso-type estimators (http://www-personal.umich.edu/~jizhu/jizhu/KnightFu-AoS00.pdf), especially for theorem 2.

The Lasso-estimator $\boldsymbol{\hat{\beta}}$ is the estimator which minimizes $$\sum_{i=1}^{n} \left( Y_i - \mathbf{x}_i^{'}\boldsymbol{\beta}\right)^2 + \lambda_n\sum_{j=1}^{p}|\beta_j|.$$ Suppose there exists a non-singular matrix $$\mathbf{C} = \lim\limits_{n \rightarrow \infty} \frac{1}{n}\sum_{i=1}^{n}\mathbf{x}_i\mathbf{x}_i^{'}.$$ For $n \rightarrow \infty$ and with the Lagrange multiplier $\lambda_n/\sqrt{n} \rightarrow \lambda_0 \geq 0 $ the authors want to show the asymptotic proporty $$\sqrt{n}(\boldsymbol{\hat{\beta}}-\boldsymbol{\beta}) \xrightarrow{d} V$$ where $$V(\mathbf{u})=-2\mathbf{u}'\mathbf{W} + \mathbf{u}'\mathbf{C}\mathbf{u} + \lambda_{0}\sum_{j=1}^{p}\left\lbrace u_j \text{sign}(\beta_j)I(\beta_j \neq 0) + |u_j|I(\beta_j=0) \right\rbrace,$$ $$\mathbf{u}=(u_1, \dots,u_p)'$$ and $\mathbf{W} \sim N(0,\sigma^2 \mathbf{C}).$

For their proof the authors claim that the function $$V_n(\mathbf{u})=\sum_{i=1}^{n} \left\lbrace \left( \varepsilon_i - \frac{\mathbf{u}'\mathbf{x}_i}{\sqrt{n}} \right)^2 - \varepsilon_i^2 \right\rbrace + \lambda_{n}\sum_{j=1}^{p}\left\lbrace |\beta_j + \frac{u_j}{\sqrt{n}}|^\gamma-|\beta_j|^\gamma \right\rbrace$$ is minimized at $\sqrt{n}(\boldsymbol{\hat{\beta}}_n - \boldsymbol{\beta})$. But I can't see why so I need some help or at least a clue.

Note: I'm only interested in case $\gamma=1$.

Theodor Ignuf
  • 11
  • It would be nice to describe more of the relevant part of the paper so that people do not need to download the paper and figure out what is going on there. – robjohn Nov 12 '14 at 16:15

1 Answers1

0

In my opinion, from equation(5) the authors have claimed that $\hat{\beta}_n$ is the minimizer of $Z_n(\Phi)$, then we get the following equation: \begin{eqnarray} \min Z_n(\Phi)&=&Z_n(\hat{\beta}_n)\nonumber\\ &=&1/n(\sum(x_i^T\beta+\epsilon_i-x_i^T\hat{\beta}_n)^2+\lambda_n\sum|\phi_i|^\gamma)\nonumber\\ &=&1/n(\sum(\epsilon_i-(\hat{\beta}_n-\beta)^Tx_i)^2+\lambda_n\sum|\phi_i|^\gamma). \end{eqnarray} Compare the last equation with $V_n(u)$, it should be easy to find that why the authors say $V_n$ is minimized at $\sqrt{n}(\hat{\beta}_n-\beta)$.

shawn Wong
  • 21