2

In the context of neural networks, I am using a function to increase the difference between "good" accuracies and "bad" accuracies, i.e, for example all accuracies below 0.8 are considered bad and all accuracies above, good.

The function looks as follow:

$$ new\_x(x) = \frac{1}{1+e^{-Ax +B}} $$ Where $x$ is the accuracy, here is the graph, with A=1 and B=0

My question is the following: how do I find the values of A and B such the function returns high values (close to 1) after $x=0.8$ and very low before $x=0.8$ (close to $0$) ?

I have tried to find it out manually, but I would like to automate the process. I guess the solution involves constraints solving, that is why I tagged it this ways (but feel free to suggest another tag).

Thanks a lot!

RobPratt
  • 45,619
RandomFellow
  • 151
  • First of all you need $\frac{B}{A}=0.8$ for the curve to change its concavity at $x=0.8$. Then for immediate change in values from $0$ to $1$ at $x=0.8$, you should take large values for $A$ and $B$, the larger, the better. For instance, you can take $A=10000, B=8000$ – Aman Kushwaha Aug 07 '21 at 07:30
  • Thanks a lot! this is very usefull, if you post the answer I will validate it. – RandomFellow Aug 07 '21 at 07:50
  • Would it be possible to also give the theory behind your solution ? (or at least a piece of it) – RandomFellow Aug 07 '21 at 07:52
  • I've written the answer consisting the theory. You can check why I suggested those values in the comment by looking at the first derivative and the point of inflection. – Aman Kushwaha Aug 07 '21 at 08:16

1 Answers1

0

Let $f(x)= \frac{1}{1+e^{-Ax +B}}$. It can be easily noticed that the range of $f$ is $(0,1)$ and for $A>0$, $f$ is strictly increasing and for $A<0, f$ is strictly decreasing.

Now, you want $f(x)$ close to $0$ when $x<0.8$ and $f(x)$ close to $1$ when $x>0.8$. That means three things: first, $A$ should be positive. Second, slope of the $f(x)$ curve should be very large at $x=0.8$. Third, the point where the graph changes its concavity from concave upwards to concave downwards should be $0.8$

You need to find $\{A>0,B\}$ such that $f'(x)_{\text{ at }x=0.8}$ is large and $f''(x)_{\text{at }x=0.8}=0$

Aman Kushwaha
  • 1,664