1

How to find the Laplacian \begin{align} \nabla^2_x f( A x) \end{align} where $n \times m$ matrix and we know know the laplacian of \begin{align} \nabla^2_y f( y) \end{align}

This question is about chain rule for the Laplacian.
I search but could not find a formula for it.

I think that the answer is \begin{align} \nabla^2_x f( A x)= {\rm Tr}( AA^T ) \nabla^2_y f( y) |_{y=Ax} \end{align}

Lisa
  • 2,941

1 Answers1

3

The gradient of the function $f(y)$ is the vector $g$ which satisfies $$df = g^Tdy$$ and the hessian is the matrix $H$ which satisfies $$dg = H\,dy$$ and the laplacian is the trace of the hessian $\,\,L={\rm Tr}(H)$.

Suppose a new independent vector $x$ is introduced with $\,y=Ax$.

It really helps to consider the case where $A$ is rectangular, so that $x$ and $y$ have different dimensions. Then when you start to wonder why $A$ was transposed or why it was pre-multiplied or post-multiplied, you'll see that it's the only way to make the dimensions come out correctly.

To find the gradient and hessian wrt $x$, one wants modifie${\rm d^*}$ quantities which satisfy $$df=g_*^Tdx,\,\,\,\,\,\,dg_*=H_*\,dx,\,\,\,\,\,\,L_*={\rm Tr}(H_*)$$ Working these out is straightforward $$\eqalign{ df &= g^Tdy = g^TA\,dx = (A^Tg)^Tdx = g_*^Tdx \cr g_* &= A^Tg \cr\cr dg_* &= A^Tdg = A^T(H\,(dy)) = A^TH(A\,dx) = H_*\,dx \cr H_* &= A^THA \cr L_* &= {\rm Tr}(A^THA) = {\rm Tr}(AA^TH) \cr }$$ The last expression is the laplacian in terms of the new variable.
Unfortunately, knowing the value of ${\rm Tr}(H)$ is of no help in evaluating ${\rm Tr}(AA^TH)$.

greg
  • 35,825
  • Thank you so much. I wanted to an example. Let $f(x)=|x |^2$. I know that $\nabla^2 f(x)=2 n$ where $n$ the dimension of $x$. Then $\nabla^2 f(A x)= 2n Tr (A A^T )$, right? – Lisa Dec 18 '18 at 15:32
  • @Lisa That's not right. Just substitute your sample function in the above formulas and you'll find $$\eqalign{ H &=2I &\implies L=2,{\rm Tr}(I) &= 2n \cr H_&=A^T(2I)A &\implies L_=2,{\rm Tr}(A^TA) &= 2|A|^2_F \cr }$$ – greg Dec 18 '18 at 17:03
  • I see now. Thank you. – Lisa Dec 18 '18 at 17:41