Suppose that second partial derivatives exist at $(a,b)$ and $\dfrac{\partial^2f}{\partial x\,\partial y}(a,b)\neq\dfrac{\partial^2f}{\partial y\,\partial x}(a,b)$.
(For example $f(x,y)=\begin{cases}xy\frac{x^2-y^2}{x^2+y^2}\quad\textrm{for } (x,y)\neq (0,0)\\0,\hspace{16mm}\textrm{for } (x,y)=(0,0)\end{cases} $)
Can function $f$ be twice differentiable at $(a,b)$ (The second-order total differential exist)?
Thank you.
Sorry for reviving this, but i was wondering about the same question today and the general case it not answered yet. So, for future visitors, the general case is answered by the following strenghtening of Schwartz's theorem:
Theorem (Schwarz) [T. M. Apostol, Mathematical Analysis (2nd ed.), Theorem 12.12] and [Hubbard, John; Hubbard, Barbara. Vector Calculus, Linear Algebra and Differential Forms (5th ed.). Matrix Editions. pp. 732–733]: Let $f:\mathbb{R}^2 \rightarrow \mathbb{R}$ be continuously differentiable on and open set around $c \in \mathbb{R}^2$ and let $\partial_x f$ and $\partial_y f$ be totally differentiable in $c$, i.e. $$\operatorname{lim}_{v\rightarrow 0} \frac{\partial_{x}f(c+v) - \partial_{x}f(c) - D(\partial_{x}f)(c)v}{||v||} = 0,$$ where $D(\partial_{x}f)(c)$ is linear. (and likewise for $\partial_y f$). Then $$\partial_{y}\partial_{x}f(c) = \partial_{x}\partial_{y}f(c).$$
proof.
Let $g(x) = f(c + (x, t)) - f(c + (x, 0))$. Then $$g(t) - g(0) = f(c + (t, t)) - f(c + (t, 0)) - f(c + (0, t)) + f(c).$$ By the mean value theorem, $$g'(\xi) = \frac{1}{t}\left( f(c + (t, t)) - f(c + (t, 0)) - f(c + (0, t)) + f(c) \right)$$ for some $0 < \xi < t$.
Since $f$ is totally differentiable in a neighbourhood of $c$, we have (for sufficiently small t) $$g'(\xi) = \frac{dg}{dx}(\xi) = \partial_x f(c + (\xi, t)) - \partial_x f(c + (\xi, 0)).$$
Because $\partial_x f$ is totally differentiable at $c$, we have
$$0 = \operatorname{lim}_{(\xi,t) \rightarrow 0} \frac{1}{\sqrt{\xi^2 + t^2}} \left( \partial_{x}f(c+(\xi, t)) - \partial_{x}f(c) - D(\partial_{x}f)(c)\cdot(\xi, t) \right).$$
Since $D(\partial_{x}f)(c) \cdot (\xi, t) = \partial_{x}\partial_{x}f(c)\xi + \partial_{y}\partial_{x}f(c)t$ and $\xi < t$, it follows that $$0 = \operatorname{lim}_{t \rightarrow 0} \frac{1}{|t|} \left( \partial_{x}f(c+(\xi, t)) - \partial_{x}f(c) - \partial_{x}\partial_{x}f(c)\xi - \partial_{y}\partial_{x}f(c)t \right) \tag{1}$$
Again, because $\partial_x f$ is totally differentiable at $c$, we have
$$0 = \operatorname{lim}_{\xi \rightarrow 0} \frac{1}{|\xi|} \left( \partial_{x}f(c+(\xi, 0)) - \partial_{x}f(c) - D(\partial_{x}f)(c)\cdot(\xi, 0) \right).$$
Again, because $\xi < t$, it follows that $$0 = \operatorname{lim}_{t \rightarrow 0} \frac{1}{|t|} \left( \partial_{x}f(c+(\xi, 0)) - \partial_{x}f(c) - \partial_{x}\partial_{x}f(c)\xi\right) \tag{2}$$
Substracting $(2)$ from $(1)$ gives us
$$0 = \operatorname{lim}_{t \rightarrow 0} \frac{1}{|t|} \left( \partial_{x}f(c+(\xi, t)) - \partial_{x}f(c+(\xi, 0)) - \partial_{y}\partial_{x}f(c)t \right).$$ Thus, for $t > 0$, $$0 = \operatorname{lim}_{t \rightarrow 0} \frac{1}{t} \left( g'(\xi) - \partial_{y}\partial_{x}f(c)t \right),$$ and we get $$\operatorname{lim}_{t \rightarrow 0} \frac{1}{t^2}\left( f(c + (t, t)) - f(c + (t, 0)) - f(c + (0, t)) + f(c) \right) =\operatorname{lim}_{t \rightarrow 0} \frac{g'(\xi)}{t} = \partial_{y}\partial_{x}f(c).$$
Defining $\tilde{g}(y) = f(c + (t, y)) - f(c + (0, y))$, we again get $$\tilde{g}(t) - \tilde{g}(0) = f(c + (t, t)) - f(c + (0, t)) - f(c + (t, 0)) + f(c).$$ So, via a similar argument, $$\operatorname{lim}_{t \rightarrow 0} \frac{1}{t^2}\left( f(c + (t, t)) - f(c + (t, 0)) - f(c + (0, t)) + f(c) \right) =\operatorname{lim}_{t \rightarrow 0} \frac{\tilde{g}'(\xi)}{t} = \partial_{x}\partial_{y}f(c),$$ hence $$\partial_{y}\partial_{x}f(c) = \partial_{x}\partial_{y}f(c).$$
The contraposition of this theorem answers the general question.
Great question. So consider the function $F\colon\mathbb R^2\to L(\mathbb R^2,\mathbb R)\cong\mathbb R^2$ given by $F(x,y) = df(x,y)$. We have $F(0,0)=0$. With a little work (or Mathematica) we find that $$F(x,y) = \begin{bmatrix} \frac{\partial f}{\partial x}(x,y) \\ \frac{\partial f}{\partial y}(x,y) \end{bmatrix} = \begin{bmatrix} \frac{y(x^4+4x^2y^2-y^4)}{(x^2+y^2)^2} \\\frac{x(x^4-4x^2y^2-y^4)}{(x^2+y^2)^2}\end{bmatrix}\,.$$
If $F$ is differentiable at $(0,0)$, of course its derivative must be given—as a bilinear form—by the $2\times 2$ matrix $$dF(0,0)=\begin{bmatrix} \tfrac{\partial^2 f}{\partial x^2}(0,0) & \tfrac{\partial^2 f}{\partial y\partial x}(0,0) \\ \tfrac{\partial^2 f}{\partial x\partial y}(0,0) & \tfrac{\partial^2 f}{\partial y^2}(0,0) \end{bmatrix} = \begin{bmatrix} 0&-1\\1&0\end{bmatrix}\,.$$
Now, if $F$ is differentiable at $(0,0)$, we must have $$\lim_{(x,y)\to (0,0)} \frac{F(x,y)-F(0,0)-dF(0,0)(x,y)}{\sqrt{x^2+y^2}} = 0\,.$$ Try it along the line $y=x$. The numerator becomes $$\begin{bmatrix} x\\-x \end{bmatrix} - \begin{bmatrix} -x\\x \end{bmatrix} = 2\begin{bmatrix} x\\-x \end{bmatrix}\,,$$ and $$\lim_{x\to 0} \frac{2 \begin{bmatrix} x\\-x \end{bmatrix}}{\sqrt 2|x|}$$ certainly is not $0$. Thus, $F$ is not differentiable at $(0,0)$.
