1

Here is my problem:

LSD

In problem (a), I thought that

$\frac{\partial}{\partial\theta_i}l_i(\theta)=(\sum_{k=1}^{p}x_{ik}\theta_k-Y_i)x_{ij}$

Thus $\nabla_{\theta}l_i(\theta)=[\frac{\partial}{\partial\theta_1}l_i(\theta) \; \frac{\partial}{\partial\theta_2}l_i(\theta) \cdots \; \frac{\partial}{\partial\theta_p}l_i(\theta)] = (X_i^T\theta-Y_i)[x_{i1}\;x_{i2}\;\cdots\;x_{ip}]=(X_i^T\theta-Y_i)X_i^T$

But as you and I already know, it's not the answer. I wonder how can I successfully explain it by using $\sum$. Where am I wrong? Please help me.

SeungJae Bang
  • 143
  • $\nabla \theta l{\color{red}i}(\theta)=\begin{bmatrix}\frac{\partial }{\partial\theta_1}l_{\color{red}i}(\theta)&\frac{\partial}{\partial\theta_2}l_{\color{red}i}(\theta)&\ldots&\frac{\partial}{\partial\theta_p}l_{\color{red}i}(\theta)\end{bmatrix}$ – PinkyWay Sep 01 '22 at 18:17
  • Oh, that's another mistake. I was wondering why the result is $(X_i^T\theta-Y_i)X_i$, which is different from my answer $(X_i^T\theta-Y_i)X_i^T$ – SeungJae Bang Sep 01 '22 at 18:29
  • $X^T=\begin{bmatrix}X_1&X_2&\ldots&X_p\end{bmatrix},$ so $X^T(X\theta-Y)=\sum_{j=1}^p(X\theta-Y)jX_j=\sum{i=1}^p(X_j^T\theta-Y_j)X_j$ – PinkyWay Sep 01 '22 at 18:29
  • I think they wrote what $\nabla\theta l_i(\theta)^T$ instead in their solution. But I may be wrong. Sorry for the typo $"i=1"$ in the last sum in my previous comment. – PinkyWay Sep 01 '22 at 18:40
  • Thank you for your help. – SeungJae Bang Sep 01 '22 at 18:55

0 Answers0