0

I have a question about the formation of weight matrix in Hopfield network: Why is not considered self-loop (i.e. $w_{ii}=0$)?

Can anyone help me in the issue? Thanks in advance!

Royana
  • 3
  • 1
    Hint: write down the energy, see how it changes when $w_{i,i}$ (for some $i$) is changed from zero to some nonzero value. Assume that the values of the nodes are $\pm 1$ – leonbloy Jan 03 '14 at 14:05
  • I have assumed a weight matrix 2*2 in which the elements of main diagonal of matrix are 2 and the others are -2. – Royana Jan 03 '14 at 14:55
  • I this case the energy becomes $E=-(x_{1}-x_2)^2$ which is negative! – Royana Jan 03 '14 at 14:56
  • while with these conditions but considering zeros in the main diagonal of weight matrix I get the Energy becomes $E=x_1*x_2$. – Royana Jan 03 '14 at 15:00
  • in this case depends on the sign of $x_1$ and $x_2$! – Royana Jan 03 '14 at 15:01
  • Now, how I can decide with these results? – Royana Jan 03 '14 at 15:02

1 Answers1

0

The energy of a configuration $S=\{s_i : s_i= \pm1 \}$is given by

$$-E(S) = \sum_{i , j} w_{ij} s_i s_j + \sum_i \theta_i s_i $$

The cuadratic sum can be decomposed in "non-diagonal" and "diagonal" terms: $$\sum_{i , j} w_{ij} s_i s_j = \sum_{i \ne j} w_{ij} s_i s_j + \sum_{i } w_{ii} \, s_i^2 = \sum_{i \ne j} w_{ij} s_i s_j + \sum_{i } w_{ii}$$

The last equality is a consequence of $s_i^2 = 1$. Then, we see that if we modify the diagonal terms $w_{ii}$, the energy is modified by a constant amount (i.e, a value that does not depend on the configuration $s_i$). But we are not interested in the absolute value of the energy, we just want to find the configuration that attains the minimum; hence, modifying $w_{ii}$ has no relevance, hence we choose to set it to zero.

leonbloy
  • 63,430
  • @ leonbloy: Thank you very much for your response. – Royana Jan 03 '14 at 18:28
  • But I have yet this problem why $w_{ii}$ is considered zero? Where do we consider the absolute value of the energy as you have mentioned? – Royana Jan 03 '14 at 18:30
  • Considering the value of zero for $w_{ii}$ can effect on stability analysis? – Royana Jan 03 '14 at 18:32
  • @Royana I have shown why $w_{ii}$ does not matter to locate the minimum. We usually assign it to zero because that's the simpler alternative (both for theoretical analysis and for numerical computation). – leonbloy Jan 03 '14 at 18:42
  • BTW: welcome to MSE, take some minutes to read the FAQ and to understand how the site works. This is not a chat/forum, long threads of comments are not welcomed. – leonbloy Jan 03 '14 at 18:46