4

Suppose I have a function $f:\mathbb{R}^n \to \mathbb{R}$. The gradient $\nabla f$ of $f$ at $x$ is the vector in the domain of $f$ satisfying $$[Df(x)] h = \langle \nabla f, h \rangle$$ for all $h$. In this case, it must be that $\nabla f = [Df(x)]^T$, so it's easy to find the gradient by just taking the transpose of the $1 \times n$ Jacobian matrix.

However, what if I use some other non-standard inner product on $\mathbb{R}^n$? Will this change my gradients? Does it change the interpretation of the gradient as the direction of steepest ascent (or is it the same interpretation, but 'direction' is being measured according to a new inner product?). Can someone please give me an example?

TheProofIsTrivium
  • 1,337

1 Answers1

2

Say your inner product is given by the matrix $(g_{ij})$, that is, for two vectors $v$, $w$ we have

$$\langle v, w \rangle = \sum_{i,j=1}^n g_{ij} v_i w_j$$

Now consider its inverse, $(g^{ij})$. We have the covector

$$d f = (\frac{\partial f}{\partial x_1}, \ldots, \frac{\partial f}{\partial x_n})$$

and the vector $\operatorname{grad} f = (a_1, \ldots, a_n)$ obtained from $d f$ by raising the indexes

$$a_i = \sum_{j=1}^n g^{ij} \frac{\partial f}{\partial x_j}$$

We see now that it is convenient to consider the components of a vector as $(a^1, \ldots, a^n)$, and also to use the Einstein summation convention. Then we can write simply

$$(\operatorname{grad} f)^{i} = g^{ij} \frac{\partial f}{\partial x^j}$$

Note that all this works for calculations on a (semi) Riemann manifold.

orangeskid
  • 53,909
  • Could you please explain what it means for an inner product to be 'given by a matrix'? – TheProofIsTrivium Sep 03 '23 at 06:32
  • @TheProofIsTrivium: added some details – orangeskid Sep 03 '23 at 06:34
  • I've never studied manifolds or differential geometry unfortunately – TheProofIsTrivium Sep 03 '23 at 06:35
  • @TheProofIsTrivium: this problem is not that difficult, the comments about Riemannian manifolds is just to show that it works in more general cases. Just take one step at a time :-) – orangeskid Sep 03 '23 at 06:55
  • I appreciate you taking the time to write this up, but I don't really understand the conclusion of your answer. Does this mean different inner products will give different gradients? – TheProofIsTrivium Sep 03 '23 at 15:45
  • Yes, they will give different gradients in general. And you have the formula for the gradients. What will not change will be the orthogonal complement of the gradient. That is the hyperplane of "zero variation". For vectors in that hyperplane, the directional derivative is $0$. This hyperplane has on one side vectors of increase of the function, on the other side vectors of decrease. You can also think that the perpendicular to the gradient is tangent to the level set through that point. – orangeskid Sep 03 '23 at 15:56