0

I was reading this article on the flipout method https://arxiv.org/pdf/1803.04386.pdf and at page $4$ the author provides the equation that describes the activations in one layer of a neural net.

$r_n$ and $s_n$ are $2$ random vectors of $\pm 1$ that gets multiplied together to create a random sign matrix as: $$y_n= \phi (W^T x_n)= \phi \biggl(\bigl(\bar{W}+ \widehat{\Delta W}\circ r_n s_n^T\bigr)^T x_n\biggr)= \phi \biggl(\bar{W}^T x_n+ \bigl( \widehat{\Delta W}^T (x_n \circ s_n)\bigr)\circ r_n \biggr) $$

where $\circ$ represent the element-wise multiplication and the subscript $n$ represent the $n^{th}$ element of the mini-batch. I do not understand what property has been used on the right side between the second and the last passage

Here is the crucial part:
enter image description here

Alucard
  • 274
  • 1
    In its present form the question will be most likely poorly received. As the users have to download the pdf and then search for the expression. Kindly consider to add that section in your question itself and if not that, at least consider attaching the screenshot of the same. Good luck! – InanimateBeing Aug 19 '22 at 17:59
  • To get help I still think you probably ought to try being more specific: Just saying "I do not understand what property has been used in the last passage" is a bit too broad. Can you try to ask a more precise question? – SBK Aug 19 '22 at 18:25
  • jesus guys hahaha , all i want to understand is how he transform the second term in the second passage so that there are now 2 $\circ$ operations with r and x switched ù – Alucard Aug 19 '22 at 18:34

0 Answers0