2

I know that backpropagation algorithm is useful to computing gradient of Error function (a scalar function) of a neural network respect to its weights.

In a paper I read that the backward pass of a neural network could be used to compute the Jacobian matrix of a function $F:\mathbb{R}^N\rightarrow\mathbb{R}^M$ (in particular it says that it is possible to compute $J_F^T\cdot v$ "propagating the vector v backward through $F$").

Unlikely I don't understand properly how can I do that.

aleio1
  • 995
  • 1
    That one: http://textlab.io/doc/4691379/-schraudolph--2002-- pag. 1729, fourth line. – aleio1 Jul 24 '16 at 16:07
  • the backpropagation algorithm works for computing the partial derivatives of any neural network function (yes a neural network is a function : giving the output in term of the output) whose graph is acyclic. That's what the author meant. A neural (non-recurrent) network is a function of the form $y_j = f_j(\sum_{i \ne j} w_{ij} y_i)$ where $W_{ij}$ are the weights of a directed acyclic graph. The input and the output are some of the $y_j$ (you know who is who by looking at the graph) – reuns Jul 24 '16 at 17:29
  • But how can I perform that? Why if i backpropagate a vector v, should i obtain the product of the transpose Jacobian matrix with v? – aleio1 Jul 25 '16 at 10:20
  • I don't think you understood the back-propagation. It computes the partial derivatives directly. What do you want more ? – reuns Jul 25 '16 at 15:21
  • @reuns where can you find closed-form statement about backprop in terms of Jacobian? – hhh Aug 30 '17 at 19:44

0 Answers0