0

I have a set of smooth functions that I need to find their maximum (not necessarily the global one, the local ones are fine too). Here are some characteristics of the mentioned functions:

  • Not known beforehand, they are generated by a program and optimized in run-time.
  • Smooth
  • Have sharp rises and falls
  • Have vast (nearly)flat areas
  • Bounded inside [0,1]

Here is an example of such function: function formula, function plot

So far, I have tested Gradient Descent on the negation of the functions to get their maximum. Sometimes it works, but its success is heavily dependent on the starting point and the gradient threshold which is going to be used as the stopping criteria. As mentioned, these functions have vast (nearly)flat areas, which every point inside them might be considered a local extermum. So a poor stopping criteria might cause the algorithm to stop at a starting point that is located inside the flat areas. Since the functions are not known beforehand and are going to be optimized in run-time, I can't set the proper stopping criteria manually.

Are there any better algorithms for my case? The ones that use the smooth characteristic of the functions are preferred.

Sachamora
  • 103
  • Golden-section search might be useful. But at any rate you may have to start with enough (equidistant, say) samples to be confident not to miss a possibly short spike ... – Hagen von Eitzen Nov 25 '17 at 10:35
  • @HagenvonEitzen I searched about the Golden-section search, it looks interesting, I certainly give it a shot. Thank you! – Sachamora Nov 25 '17 at 11:15
  • Gradient descent is likely to be quite bad. Is it cheap to calculate a Fourier series approximation? – mathreadler Nov 25 '17 at 12:58
  • Have you tried stochastic gradient descent (SG)? The randomness or noise introduced by SG allows to escape from local minima to reach a better minimum. – Joey Castro Nov 30 '17 at 20:47

0 Answers0