How did this gradient get derived?

Asked Jun 12 '23 at 13:05

Active Jun 13 '23 at 13:27

Viewed 44 times

I am reading Pattern Recognition and Machine Learning by Bishop and equation 6.2 for gradient of regularized least squares is

$$J(w) = \frac{1}{2}\sum^{N}_{n=1}\{\mathbf{w^T}\phi(\mathbf{x_{n}})-t_{n}\}^2+\frac{\lambda}{2}\mathbf{w^Tw}$$

then in the next step it says taking the gradient with respect to w leads to:

$$w = -\frac{1}{\lambda}\sum_{n=1}^{N}\{\mathbf{w^T}\phi(\mathbf{x_{n}})\}\phi(\mathbf{x_{n}})$$

but I don't understand where the $-\frac{1}{\lambda}$ is coming from?

wouldn't the differential be:

$$\lambda\mathbf{w} + \sum_{n=1}^{N}\{\mathbf{w^T}\phi(\mathbf{x_{n}})\}\phi(\mathbf{x_{n}})$$

edited Jun 13 '23 at 13:27

asked Jun 12 '23 at 13:05

amstergc20

1

They want the gradient to be zero. Use your differential, put $\lambda\mathbf{w}$ on the left hand side and divide by $\lambda,.$ – Kurt G. Jun 12 '23 at 13:12
Of course. too much time staring at the step when the answer is right in front. Thanks! – amstergc20 Jun 12 '23 at 13:15
1

Can you give a better reference than "Pattern Recognition and machine learning"? – user1729 Jun 13 '23 at 09:47

0 Answers0