1

In machine learning there is a function called 'oneplus':

$$y = 1+ ln(1+x)$$

Edit: this is actually a wrong formula. For the correct one, see the comment under this question, and the accepted answer.

The derivative in terms of input is

$$ \frac{dy}{dx} = \frac{e^x}{1+e^x} $$

But, how can I compute the derivative in terms of function's output? This would let me store the original result of the function in code, and use it to compute derivative (instead of using $x$). It's similar to this question

I am trying to use implicit differentiation as follows:

$$e^y = e^{1+ln(1+x)}$$ $$e^y = e \cdot e^{ln(1+x)}$$ $$e^y = e \cdot (1+x)$$ $$\frac{\mathrm{d} e^y}{\mathrm{d}x}= \frac{\mathrm{d}}{\mathrm{d}x}e(1+x)$$ $$\frac{\mathrm{d} e^y}{\mathrm{d}x} = e$$ $$\frac{\mathrm{d} e^y}{\mathrm{d}x} \cdot \frac{\mathrm{d}y}{\mathrm{d}y} = e$$ $$\frac{\mathrm{d} e^y}{\mathrm{d}y} \cdot \frac{\mathrm{d}y}{\mathrm{d}x} = e$$ $$e^y \cdot \frac{\mathrm{d}y}{\mathrm{d}x} = e$$ $$\frac{\mathrm{d}y}{\mathrm{d}x} = \frac{e}{e^y}$$

However, the correct formula is: $$\frac{\mathrm{d}y}{\mathrm{d}x}=\frac{e^y -e}{e^y}$$

Where did I make the mistake?

Kari
  • 229

1 Answers1

2

Let's assume the correct form of the function is $y=1+\ln(1+e^x)$ instead. Then we can avoid implicit differentiation by noticing

$$e^{y-1} = 1 + e^x$$

and plugging that relationship into the derivative

$$\frac{dy}{dx} = \frac{e^x}{1+e^x} = \frac{e^{y-1}-1}{e^{y-1}} = \frac{e^y-e}{e^y}$$

Ninad Munshi
  • 34,407