0

Refer http://www.deeplearningbook.org/contents/mlp.html Page 207

Usually we do not apply the back-propagation algorithm merely to vectors, but rather to tensors of arbitrary dimensionality. Conceptually, this is exactly the same as back-propagation with vectors. The only difference is how the numbers are arranged in a grid to form a tensor. We could imagine flattening each tensor into a vector before we run back-propagation, computing a vector-valued gradient, and then reshaping the gradient back into a tensor. In this rearranged view, back-propagation is still just multiplying Jacobians by gradients.

I don't understand the above statement from a book.

  1. What does it mean by flattening each tensor to a vector, why would you do that? What is a tensor here,- a vector, matrix or more than 2 dimensions, and how does that affect the process of flattening?
  2. What does the last statement mean?
Abhishek Bhatia
  • 1,648
  • Which book? $\phantom{}$ – Angina Seng Jun 22 '17 at 04:01
  • @LordSharktheUnknown added the book and page. – Abhishek Bhatia Jun 22 '17 at 04:06
  • 2
    What book? What context? "back-propagation algorithm" sounds 'computer-ish'. I would hope some of the terms are defined in the book. "Flattening a tensor into a vector" might be a generalization of the process of placing each row of a matrix in the same ordered $nm-$tuple. So the $3\times3$ matrix is {{a,b,c},{d,e,f},{g,h,i}}, then the flattened version is the 3x3-tuple {a,b,c,d,e,f,g,h,i}. Sorry, I am posting from my phone so I'm not using good notation. – Steven Thomas Hatton Jun 22 '17 at 04:16

0 Answers0