1

I need to calculate the following enter image description here

which is Shannon Entropy formula only using simple functions like log or squareroot or I don't know, just make it simple enough for a guy that does understand only grammar school math, if that is possible, so I don't understand this at all. For starters I don't even know what to do with the "big E" in the beginning of each part of this formula, that is how I "understand" math :(

Vlastimil Burián
  • 125
  • The big E (Sigma, $\Sigma$) means sum over all $i$. $\log(1/x) = -\log(x)$ is a well known identity of the logarithm. Would you know how to calculate the entropy given the $x_i$? – mlainz Apr 30 '16 at 04:23
  • Or do you want to understand why do we use that formula? – mlainz Apr 30 '16 at 04:25
  • The formula looks a little bit scary, but it's actually very simple. – mlainz Apr 30 '16 at 04:38

1 Answers1

1

For example, let $b = 2$. You have 3 events $x_1, x_2, x_3$,

$$p(x_1) = 0.2\\ p(x_2) = 0.5 p(x_3) = 0.3$$

Then $$H = -\sum_i p(x_i) \log_2(p(x_i)) = - p(x_1) \log_2(p(x_1))- p(x_2) \log_2(p(x_2))- p(x_3) \log_2(p(x_3)) =\\ 0.46 + 0.50 +  0.52 = 1.48 $$

Perhaps, if you tell me in which context you found that formula, I could give you a more meaningful example.

mlainz
  • 1,275