0

If we apply Shannon's formula to a sequence of numbers that we know how to generate (for example, natural numbers), shouldn't entropy be 0?

Mine might be a too intuitive definition of entropy. But if something is predictable, does it have entropy at all? Notwithstanding this is the fact that it appears to have entropy. However, is '000000' any different from '012345'? According to online entropy calculators it is.

Pierre B
  • 119
  • Presumably, everybody knows how to generate the messages he sends, so shouldn't the entropy of every message be $0?$. – saulspatz Jun 30 '18 at 13:07
  • @saulspatz: Not every message is sent by someone who knows how to generate it. "Knowing how to generate a message" sounds like it would require the message to be computable; so almost no infinite messages can be sent by someone who knows how to generate them :-) – joriki Jun 30 '18 at 14:07
  • 1
    @joriki I was being facetious of course, but I was thinking of ordinary communication, as was Shannon -- although he was a lot better at thinking than I am. – saulspatz Jun 30 '18 at 14:10
  • You have to take limits to define the entropy for a sequence whose domain is infinite, like the sequence (0,1,2,3...n-1,n,n+1....). What definition are you using for that? – Raúl Aparicio Bustillo Mar 28 '20 at 12:37

0 Answers0