According to the Gauss-Newton algorithm, it was used to minimize the sum of square loss $f = \dfrac{1}{2}r^2$ where $r$ is the residual and it can be written as $r = y-f(x)$.
The normal equation of Gauss-Newton is $J^TJx=-J^Tr$ where $J$ is known to be the Jacobian matrix of the residual $r$. However, Gauss-Newton use the least square loss which makes the outliers take over the minimization. There are many loss function designed to reduce the effect of these outliers, such as Huber, Tukey, Cauchy, etc.
The Huber formula is listed as $$\operatorname{Huber}(r) = \begin{cases} \tfrac{1}{2}r^2 & \text{if } |r| < c,\\ c(|r|-\frac{c}{2}) &\text{otherwise.}\\ \end{cases}$$ Huber function takes residual as input. If the absolute value of $r$ is smaller than the constant $c$, Huber function acts as the least square loss and I know know to solve with Gauss-Newton. But what really bother me is the other case $c(|r|-\frac{c}{2})$.
Question 1. How to calculate the Jacobian when $|r|\geqslant c$?
Here is my trial.
$$J_\text{otherwise} = \frac{\partial\operatorname{Huber}(r)}{\partial x} =
\frac{\partial\operatorname{Huber}(r)}{\partial r} \frac{\partial r}{\partial x} =
\operatorname{sign}(r)c \cdot \frac{\partial r}{\partial x}$$
Question 2. What is the normal equation of Gauss-Newton with Huber function?
Here again, I give my thought:
$$\text{Gauss-Newton} =
\begin{cases}
J^TJx=-J^Tr & \text{if } |r| < c,\\
J^T_\text{otherwise}J_\text{otherwise}x=-J_\text{otherwise}^Tr &\text{otherwise.}\\
\end{cases}$$
It seems that no matter what loss function I use, residual $r$ is always $y-f(x)$, right?
