0

I am searching for a function to show the "zig-zag" pattern of the steepest decent, I want to show that SGD converges after less iterations.

Which functions should look at?

newhere
  • 3,115

1 Answers1

1

Steepest Descent (gradient descent with line search) exhibits zigzag behavior on the Rosenbrock Banana function.

$(x-1)^2 + 100(y-x^2)^2$ using the starting point (-1.2,1). Lots of zigzags to get to the optimum at (1,1). You can make it zigzag even more by increasing 100 to a larger number.

Mark L. Stone
  • 4,496