0

I was reading an article about image processing and I came across sigmoidal activation function and tanh like in this article:

enter image description here

But I'm struggling to understand the concept behind the nonlinearity here and what has to do with tanh, could anyone clarify this idea, please

1 Answers1

0

$\psi(t)=\mathrm{tanh}(\alpha t)$ is a nonlinear transformation.

For $|\alpha t|$ close to $0$, the function is actually an approximately linear function of $t$, with $\psi(t) \approx \alpha t$.

By the time $|\alpha t| \ge 2$, the function's nonlinearity is very apparent, almost not really depending on $\alpha t$ at all, but just the sign of $\alpha t$, so $\psi(t) \approx \mathrm{sgn}(\alpha t)$.

In summary, $\psi(t)=\mathrm{tanh}(\alpha t)$ is a nonlinear tranformation with a "softer" transition than the basic $\psi(t)=\mathrm{sgn}(t)$ nonlinear transformation.

Andy Walls
  • 3,461
  • It seems logic what you said but why 2? –  May 15 '18 at 09:49
  • See the plots here: http://m.wolframalpha.com/input/?i=tanh%28x%29 – Andy Walls May 15 '18 at 09:52
  • Okay I see: 2 depends on $\alpha$ but what does represents this value to tanh? –  May 15 '18 at 09:54
  • 1
    According to the excerpt, tanh() is used as part of a blob detector in images. tanh() turns darker portions of an image into a -1, and lighter portions into a +1, so you can classify light regions and dark regions. By using tanh() you can also classify regions that transition between light and dark. – Andy Walls May 15 '18 at 10:13