0

I understand in the gradient descent optimisation algorithm , in each iteration we need to take a step opposite to the direction of the gradient. enter image description here

In the above diagram the direction of the gradient vector is opposite to delta theta? Can somebody please help me understand this?

Ayan Biswas
  • 113
  • maybe trying to minimize the cost function that's why opposite direction – emonHR Dec 26 '19 at 07:54
  • but the step has to be taken in the direction of delta theta . so how is it in opposite direction of the gradient? – Ayan Biswas Dec 26 '19 at 07:59
  • In the above diagram, $\Delta \theta$ is a scalar and the gradient at a given point $\theta \in \mathbb R$ is also a scalar. That arrow pointing down and to the left is unhelpful and potentially misleading. If this loss function $f$ has its minimum at $\theta=0$, then $\nabla f(\theta)$ visually points to the right if $\theta$ is positive and points to the left if $\theta$ is negative. – littleO Dec 26 '19 at 08:02
  • 1
    There are only two directions in $\mathbb{R}$, these are $\pm 1$. – copper.hat Dec 26 '19 at 08:52

0 Answers0