Is the spectral norm of a matrix differentiable?

Question

Let $X$ be an $m\times n$ matrix and let $f(X) = \|X\|$ denote its spectral norm. Does $\frac{\partial f(X)}{\partial X_{ij}}$ always exist?

Answer 1

It exists iff the largest singular value is simple and not zero. This is due to the fact that the spectral norm is the largest singular value, which is, in turn, the square root of the largest eigenvalue of $X^\top X$.

And the largest eigenvalue is differentiable w.r.t. the entries of the matrix iff it is simple.