0

In this video on Machine Learning, https://www.youtube.com/watch?v=ASn7ExxLZws , the lecturer suggests that it is good to try to prove this:

$$\sigma(-x) = 1 - \sigma(x)$$

Where: $$\sigma(x) = \frac{1}{1 + e^{-x}}$$

I produced this proof. It is a very long time since I did mathematics.

Substitute in the sigmoid formula: $$\frac{1}{1 + e^x} = 1 - \frac{1}{1 + e^{-x}}$$

Minus 1 from both sides: $$\frac{1}{1 + e^x} - 1 = - \frac{1}{1 + e^{-x}}$$

Substitute: $e^{-x} = \frac{1}{e^x}$: $$\frac{1}{1 + e^x} - 1 = - \frac{1}{1 + \frac{1}{e^x}}$$

Multiply both sides by $1 + \frac{1}{e^x}$: $$\frac{1 + \frac{1}{e^x}}{1 + e^x} - 1 - \frac{1}{e^x} = -1$$

Add 1 to both sides: $$\frac{1 + \frac{1}{e^x}}{1 + e^x} - \frac{1}{e^x} = 0$$

Add $\frac{1}{e^x}$ to both sides: $$\frac{1 + \frac{1}{e^x}}{1 + e^x} = \frac{1}{e^x}$$

Multiply both sides by $1 + e^x$: $$1 + \frac{1}{e^x} = \frac{1 + e^x}{e^x}$$

Multiply both sides by $e^x$: $$e^x + 1 = 1 + e^x$$

Is this correct?

TTransmit
  • 103
  • Yes, it seems fine. Equivalently, you could directly evaluate $\sigma(x)+\sigma(-x)$ and conclude. $$\small\sigma(x)+\sigma(-x)=\frac 1{1+e^{-x}}+\frac 1{1+e^x}=\frac{2+e^x+e^{-x}}{(1+e^x)(1+e^{-x})}=\frac{1+e^x+e^{-x}+e^xe^{-x}}{(1+e^x)(1+e^{-x})}=\frac{(1+e^x)(1+e^{-x})}{(1+e^x)(1+e^{-x})}=1$$ – Prasun Biswas Sep 07 '17 at 18:12

1 Answers1

5

Yes, that's fine, but a bit long. Shorter:

$$\begin{align}\sigma(x)&=\frac{1}{1+e^{-x}}\\ &=\frac{e^x}{e^x+1}\\ &=\frac{e^x+1-1}{e^x+1}\\ &=\frac{e^x+1}{e^x+1}-\frac{1}{e^x+1}\\ &=1-\sigma(-x)\end{align}$$

This is the same as $\sigma(-x)=1-\sigma(x)$.

Your proof is somewhat backwards, because you've assumed your result then proved that it implies a true equality. But you really need to make sure the reverse works - that $1+e^x=e^x+1$ implies your desired equality. Here, you can reverse your argument, but it is always important to check that you can reverse the argument.

Thomas Andrews
  • 177,126