I need to compute the following: $\frac{\partial}{\partial y} \left( \frac{ dy}{dx} \right)$. I understand that this looks funny and that it's not quite clear why I would want to compute that. The reason is that I will be optimizing something of the form:
$$I[y] = \sum_k h\left( \frac{dy}{dx}(x_k) \right)$$
or
$$I[y] = \int h\left( \frac{dy}{dx} \right) dx$$
I attempted to use the chain rule and found, for $g(y(x)) := \frac{dy}{dx}$, that:
\begin{align}\frac{d^2 y}{dx^2} & = \frac{d}{dx} \frac{dy}{dx} \\ & = \frac{d}{dx} g(y(x)) \\ & = \frac{\partial g}{\partial y} \frac{dy}{dx} \\ &= \frac{\partial}{\partial y}\left( \frac{dy}{dx} \right) \frac{dy}{dx} \end{align}
This allows me to solve for what I wanted and obtain:
$$\frac{\partial}{\partial y}\left( \frac{dy}{dx} \right) = \frac{ \frac{d^2 y}{dx^2}}{\frac{dy}{dx}}$$
However, when I think about this numerically, I would compute a sampled version of $\frac{dy}{dx}$ using some linear finite-difference operation, e.g. $\frac{dy}{dx} \approx Dy$, where $D$ is, say, a first-order centered difference operator. Then, I can compute the gradient of that with respect to $y$ and it is simply $D^T$ (an operator!).
So, I have two conflicting approaches and neither seems to be very clear. Does anybody have a clear idea of what's going on here? Thanks in advance.
