1

If the directional derivative of a function $g$ exists for any $\boldsymbol{\theta}$ in any direction ${\bf d}$, and can be expressed as:

\begin{equation} \lim_{\tau \rightarrow 0} \frac{g(\boldsymbol{\theta} + \tau {\bf d}) - g(\boldsymbol{\theta})}{\tau} = {\bf \Delta}^\top {\bf d} \end{equation} where ${\bf d}$ denotes any direction and $\tau$ is a scalar.

My question is: does this mean $g(\cdot)$ is differentiable at everywhere? And does this imply $\nabla g(\boldsymbol{\theta}) = \Delta$?

My understanding is that for any differentiable function $f(\cdot)$, its directional derivative $\nabla_{\bf d} f(\boldsymbol{\theta})$ can be expressed as $\nabla_{\bf d} f(\boldsymbol{\theta}) = \nabla^\top f(\boldsymbol{\theta}){\bf d}$, where $\nabla f(\boldsymbol{\theta})$ is the gradient of $f$ evaluated at $\boldsymbol{\theta}$. The first equation means the directional derivative of $g(\boldsymbol{\theta})$ can be written in the same form of a differentiable function. So does this implies $g(\boldsymbol{\theta})$ is differentiable at any $\boldsymbol{\theta}$, and its gradient is $\nabla g(\boldsymbol{\theta}) = \Delta$? I can not guess any function which is not differentiable but its directional derivative can be written as $\Delta^\top {\bf d}$, for a constant vector $\Delta$ and any vector ${\bf d}$.

Can anyone help me? Thank you in advance!

uniquesolution
  • 18,541
Harry_Cai
  • 45
  • 1
    The difference is that around the (0,0), can it be expressed as $\lim_{\tau\rightarrow 0 }\frac{g(\tau {\bf d})}{\tau} = \Delta^\top d$ – Harry_Cai Jan 30 '23 at 07:40
  • Contrarily to uniquesolution's answer below, @RyszardSzwarc 's answer in the duplicate gives the counterexample you asked for. – Anne Bauval Jan 30 '23 at 13:51
  • @Anne Bauval -- You are wrong. The formula assumed to hold everywhere by the OP does not hold everywhere in the example given by RyszardSzwarc, which is c classical example as such. – uniquesolution Feb 19 '23 at 09:46
  • @uniquesolution You are right, in @RyszardSzwarc 's answer the formula only holds for $\theta=(0,0).$ From this point of view, your answer below, even after you corrected it, is not better. – Anne Bauval Feb 19 '23 at 11:04

2 Answers2

2

It isn't too difficult to find an example where the formula \begin{equation} \lim_{\tau \rightarrow 0} \frac{g(\boldsymbol{\theta} + \tau {\bf d}) - g(\boldsymbol{\theta})}{\tau} = {\bf \Delta}^\top {\bf d} \end{equation} holds for a specific $\theta$ and every ${\bf d}$, but the function is not even continuous at the point $\theta$. For example, consider the set $$S=\{(x,y):\ \ 0<y<x^2\}$$ and its characteristic function $f(x,y)=\chi_S(x,y)$. Then with $\theta=(0,0)$ and every ${\bf d}$ and sufficiently small $|\tau|$ the difference $f(\theta+\tau{\bf d})-f(\theta)$ vanishes, hence all directional derivatives exist at the point $\theta=(0,0)$ and are equal to zero there, so the formula holds. However, the function is clearly not continuous at $\theta=(0,0)$.

This is only a partial answer, of course. I don't know whether the assumption that the formula above holds for every $\theta$ causes the result to be different. At any rate, it is an interesting question.

Contrary to some comments by Anne Duval, this is not a duplicate question and the first link offered as answering your question does not answer your question, as my previous answer did not. It was merely an example where all partial derivatives exist but the function is not differentiable. It is irrelevant because even in those examples the formula above does not hold.

Edited - the comments below are now irrelevant. They refer to my previous answer. They are however relevant to the link offered in the first comment above.

uniquesolution
  • 18,541
  • 1
    The limit in the question for this example turns into $\tfrac {(d_x \tau)^3}{\tau((d_x \tau)^2+(d_y \tau)^2)}$ which reduces to $\tfrac {d_x^3}{d_x ^2+d_y ^2}$ which can't be expressed as $\Delta^T d$ – Alex K Jan 30 '23 at 01:47
  • 1
    I agree, I am not sure if the difference is that around the (0,0) can be expressed as $\lim_{\tau\rightarrow 0 }\frac{g(\tau {\bf d}) - g(0)}{\tau} = \Delta^\top d$ – Harry_Cai Jan 30 '23 at 07:43
1

The function $$(x,y)\mapsto\begin{cases}0&\text{if }|y|\ge x^2 \\ \left(\frac{y}{x^2}\left(1 - \frac{y}{x^2}\right)\right)^2&\text{if }|y| \le x^2 \end{cases}$$ is differentiable everywhere except at $\theta=(0,0)$ and at that point, it is discontinuous but has directional derivatives equal to $0$ in every direction ${\bf d}.$

The proof is the same as for this variant.

Anne Bauval
  • 34,650