I'm trying to prove that $f$, defined as $$f(x,y)=\left\{\begin{array}{cl} \dfrac{x^2y}{x^2 + y^2}&(x,y)\neq (0,0) \\ 0&(x,y)=(0,0)\end{array}\right. ,$$ is not differentiable at $(0,0)$. Is the correct way to approach such problems just to show that $$\lim_{(h,k)\to(0,0)}\frac{f(h,k)-f(0,0)-\mathbf{A}(h,k)}{\sqrt{h^2+k^2}}\neq0\ ?$$ Or should I show that that limit does not exist?
I was told that even if the limit turns out to be non-zero, even then the function will be differentiable at $(0,0)$. Is that true? I mean if the limit doesn't turn out to be $0$ doesn't it mean that the tangent plane is not the best linear approximation for the function? If tangent plane isn't the best linear approximation, how can it be differentiable at that point (given the fact that the tangent plane is defined as the derivative)?
Also, should I take $A$ as just the jacobian $[\frac{\partial f}{\partial x} \space \frac{\partial f}{\partial y}]$ and $A(h,k)$ as $[\frac{\partial f}{\partial x} \space \frac{\partial f}{\partial y}][h \space k]^T$ ?