I want to have a list of 5 learning rates. My min LR is 1e-6 and my max LR is 1e-4. I want to have 3 LRs in between so that they are evenly distributed.
LRs = [1e-6, ..., ..., ..., 1e-4]
Sorry if this is a very elementary question but what could be those 3 values to have evenly distributed 5 LRs where they all step-up to 1e-4. These are going to be my 5 LRs for fine-tuning my model.
The difference between any two adjacent values is $$\frac{10^{-4}-10^{-6}}{5-1}=0.00002475. \tag{1}$$ Therefore, the 5 learning rates should be \begin{align} \mathrm{LRs} = & [10^{-6}, 10^{-6} + 0.00002475, 10^{-6} + 0.00002475 \times 2, 10^{-6} + 0.00002475 \times 3, 10^{-6} + 0.00002475 \times 4, 10^{-4}] \\ = & [10^{-6}, 0.00002575, 0.0000505, 0.00007525, 0.0001]. \end{align}
You are going to have five monotonically increasing values (i.e in order), this creates 4 regions--the thing you actually need to compute. And that's your answer, the width of each region is simply:
$$ \frac{10^{-4} - 10^{-6}}{4} = 2.475\times10^{-5} $$
