2

It seems to me that many iterative methods in numerical analysis leverage some sort of local contraction to show that a solution will converge to a fixed point. Is this true of all iterative methods or are there iterative methods which do not require local contraction?

awright96
  • 425
  • Depends on how strictly you define your terms - basically, if $a^$ is the solution to your problem, and $a$ is some guess of the solution, then each step of the iterative method ought to bring you closer to $a^$. "Getting closer" is (again, depending on your definition) arguably equivalent to "the method is contractive around $a^*$. – πr8 Apr 18 '17 at 18:43

1 Answers1

1

Something interesting in this direction may be the fact that Nesterov's accelerated gradient descent is known for occasionally increasing your function error. Here is an example with some pictures of that and discussion.

amakelov
  • 3,520
  • 12
  • 25