2

I read somewhere that if $F:U\in \Bbb{R}^n\to \Bbb{R}^m$ is differentiable at $X$ (where $X\in U$), then there exists an $m \times n$ matrix $A$ such that $$F(X+h)=F(X)+A\circ h+G(h)\|h\|$$ where $G(h)\to 0$ as $h\to 0$ (and as $\|h\|\to 0$). $A$ here is the matrix $F'(X)$.

What does this mean? I only see a weak analogy to differentiation in the form $\lim_{h\to 0}\frac{F(X+h)-F(X)}{h}$. I'm really having problems understanding this.

I'm given to understand that $A\circ h$ is the product of $A$ and $h$ where $h$ is an $n\times 1$ vector belonging to $\Bbb{R}^n$, and that $F(X)+A\circ h$ is the linear approximation of $A(X+h)$ while $G(h)\|h\|$ is the correction factor. Does this have a proof?

Thanks in advance!

  • Whether you're doing single variable or multivariable calculus, the best way to think of the derivative is like this: $f(x + \Delta x) \approx f(x) + f'(x) \Delta x$. – littleO Aug 26 '13 at 22:02

1 Answers1

0

This formula is related to the definition of differentiability. Differentiability of functions $f:\mathbb{R}^{n}\to\mathbb{R}^{m}$ means that there exists an $m\times n$ matrix $A_{x}$ such that

$\lim_{h\to0}\frac{f(x+h)-f(x)-A_{x}h}{\|h\|}=0$.

Define $G(h)=\frac{f(x+h)-f(x)-A_{x}h}{\|h\|}$. Then

$f(x+h)=f(x)+A_{x}(h)+G(h)\|h\|$ where $\lim_{h\to0}G(h)=0$. This is how you justify the formula. The definition of $G$ that I gave shows that it is a correction factor that measures how far off $f(x+h)$ is from the approximation $f(x)+A_{x}h$.

user71352
  • 13,038
  • @user71352- What is the motivation behind stating differentiability implies $\displaystyle{\lim_{h\to 0}\frac{f(x+h)-f(x)-A_x h}{|h|}=0}$? Wouldn't defining it as $\displaystyle{\lim_{h\to 0}\frac{f(x+h)-f(x)}{|h|}=A_x}$ make more sense from the intuition perspective, considering $A_x$ is $f'(x)$? –  Aug 26 '13 at 22:10
  • I stated the definition of differentiability with the limit since I knew I would have to later demonstrate that $\lim_{h\to0}G(h)=0$. You can'd define differentiability like that in general. $A_{x}$ is an $m\times n$ matrix but $\frac{f(x+h)-f(x)}{|h|}$ is a $m\times 1$ matrix for all $h$. – user71352 Aug 26 '13 at 22:25
  • Note that $\lim_{h\to0}\frac{f(x+h)-f(x)}{||h||}$ is the directional derivative of $f$ in the direction given by $h\in\mathbb{R}^n$. As such, $\lim_{h\to0}\frac{f(x+h)-f(x)}{||h||}$ strongly depends on how the vector $h$ approaches the origin: it does not have a unique value. (user71352 alluded to this fact by mentioning that the result is an $m\times1$ matrix - indeed, the directional derivative.) $A_x$ as defined above, on the other hand, is the matrix collecting all such directional derivatives. – automaton 3 Aug 26 '13 at 23:00