I'm using the Nelder-Mead method to minimize a quantity with 4 variables for a given input. I want to know how I should calculate the confidence interval, ie. to answer the question: with what confidence am I getting the true minimum?
The calculation of the quantity is incredibly complex with lots of lines of code and cannot be modeled with a simple function approximation technique, so it can't be benchmarked with a function approximation.
My thoughts so far: The Nelder-Mead method doesn't guarantee convergence all the time, and depends on the starting point in the space. So far, I'm trying to minimise the quantity by running the algorithm for different starting points across the space.
Eg. for the 4 variables, my starting points look like [0,0,0,0], [2,2,0,0],[-2,-2,-2,-2],[0,0,-2,-2] etc., basically the vertices of a hypercube. I run the algorithm for each one, and take the overall minimum as my optimal minimum.
To calculate the confidence interval, I thought I'd assume a normal distribution, and calculate its mean by taking the average of what each of those algorithm runs gives. I don't know if this is accurate or how to determine standard deviation from this.
