Not understanding some derivatives for e (ML error function)

Question

In an Udacity machine learning class it is showing the following simplification.

\begin{align} \sigma'(x) & = \frac{\partial}{\partial x} \frac{1}{1+e^{-x}} \\ & = \frac{e^{-x}}{(1+e^{-x})^2} \\ & = \frac{1}{1+e^{-x}} \cdot \frac{e^{-x}}{1+e^{-x}} \\ & = \sigma(x)(1-\sigma(x)) \end{align}

(Original at https://i.stack.imgur.com/9ihZ2.gif)

I am not following the line 1 to 2 and line 3 to 4. For line 1 -> 2, shouldn't that now be 1 / ( - (1 + e^x))?

The for line 3 -> 4, the sigmoid is 1 / (1 + e^x) and so I understand the sigmoid(x) part of the equation. But how does e^x/(1+e^x) become (1 - sigmoid(x))? Shouldn't it be e^-x / sigmoid(x)?

(I knew all this real well 40 years ago when I earned a math degree, but 40 years of never touching this stuff and all that knowledge is gone...)

Answer 1

From line $1$ to line $2$: This is an application of the chain rule. We have

\begin{align} \sigma'(x) & = \frac{d}{dx} \frac{1}{1+e^{-x}} \\ & = \frac{d}{dx} (1+e^{-x})^{-1} \\ & = - (1+e^{-x})^{-2} \frac{d}{dx}(1+ e^{-x}) \\ & = - (-e^{-x}) (1+e^{-x})^{-2} \\ & = \frac{e^{-x}}{(1+e^{-x})^2} \end{align}

From line $3$ to line $4$: This is a recognition that

\begin{align} \frac{e^{-x}}{1+e^{-x}} & = \frac{1+e^{-x}-1}{1+e^{-x}} \\ & = 1 - \frac{1}{1+e^{-x}} \\ & = 1 - \sigma(x) \end{align}