Mathematics Stack Exchange
Mathematics Stack Exchange

Questions tagged [entropy]

This tag is for questions about mathematical entropy. If you have a question about thermodynamical entropy, visit Physics Stack Exchange or Chemistry Stack Exchange instead.

This tag is for questions about mathematical entropy, not to be confused with thermodynamical entropy which goes in Physics or Chemistry Stack Exchange.

1617 questions
2
votes
1 answer

Maximum value of Kullback–Leibler entropy

The measure of Shannon Entropy should be maximal if all the outcomes are equally likely (uncertainty is highest when all possible events are equiprobable). I.e., as we can see in Wikipedia: $$\mathrm {H} _{n}(p_{1},\ldots ,p_{n})\leq \mathrm {H}…
Mark
  • 7,841
  • 6
  • 38
  • 72
2
votes
0 answers

What is weighted entropy?

I got this question but I don't understand what 'weighted entropy' mean here ? Age is the x-axis and Salary is the y-axis. Compute the following: The entropy of the full set of points. The entropy of the set of points that belong to the left child…
chickensoup
  • 155
2
votes
2 answers

Showing that one entropy is greater than another

There is two fair (identical) coins. Heads is worth one point and tails is worth two points. We flip two coins at a single time. Lets consider the two experiments $X$ & $Y$ on the set $S=$ {2, 3, 4}. Experiment $X$ the set $P_x(x)$ is the…
Temirzhan
  • 983
  • 1
  • 11
  • 25
2
votes
1 answer

Password entropy of famous xkcd comic

The famous xkcd comic about password strength calculates the entropy of the 11-character password "Tr0ub4dor&3" with 28 bits of entropy. When following the ASCII-95-chart, we have 95 possible letters, numbers and symbols for each character…
sqe
  • 23
2
votes
1 answer

Can topological entropy be infinte?

I wonder if the topological entropy as defined by Adler or Bowen can be infinity. Can you answer that?
Salamo
  • 1,094
2
votes
1 answer

Topological Entropy of $T$, on a disjoint union?

Let $X$ be a compact metric space and $T\colon X\to X$ continuous. By $h(A\cup B\cup C,T_{|A\cup B\cup C})$ denote the toplogical entropy of $T$, restricted on $A\cup B\cup C$, where $A,B,C\subset X$ are disjoint. Is then $$ h(A\cup B\cup…
Salamo
  • 1,094
2
votes
0 answers

Topological Entropy of $T$ on subset $Y\subset X$

Let $X=\left\{0,1,2\right\}^{\mathbb{Z}}$ and on it the following dynamics described by $T\colon X\to X$ as follows: A 1 becomes a 2, a 2 becomes a 0 and a 0 becomes a 1 if at least one of its two neighbors is 1. Moreover,…
user34632
2
votes
0 answers

Is it true that $\sup_{y'\in Y'}h_d(T,U^{-1}(y'))=0$, i.e. $h(T')=h(T)$? (Bowen, Topological entropy)

First I have to give the background to my question: Let $X=\left\{0,1,2\right\}^{\mathbb{Z}}$ and on it the map $T\colon X\to X$ which describes the following dynamics: For $x\in X$, which is a bi-infite sequence, let $x(i)$ denote the i-th…
user34632
2
votes
1 answer

Two definitions of topological entropy: Why do they coincide?

I guess you all know the definition of topological entropy by using open covers for $X$ being a compact topological space and $T\colon X\to X$ being a continuous map (for example given in Walters' "An Introduction to Ergodic Theory"); here is a…
user34632
1
vote
0 answers

Minimum of Normalized Joint Entropy

I know that if we want to normalize variable $V$, we should do the following: $(V-\min)/(\max - \min)$ So, for Normalizing Joint Entropy $H(x,y)$ , this should be correct: $$ \frac{H(x,y)-\max(H(x),H(y))}{H(x)+H(y)-\max(H(x),H(y))} \tag{1} $$ The…
A.Ra
  • 11
1
vote
1 answer

Computing entropies of designs.

I've been told to assume $6$ coins are weighed on a chemical balance (two-pan) scale. We are told exactly two coins are fake and that fake coins are heavier than real ones. I've been given two designs of experiments that were used to determine the…
Sean E.
  • 23
1
vote
2 answers

Shannon Entropy Question in C# (but mainly an algebra question)

I'm looking at someone else's project (in c#) and there's an entropy calculation I don't understand. I haven't done any maths in ages and I've had to look up entropy and Log in pursuit of this, so I'm happy to have gotten this far but now I'm…
samuelthomson
  • 25
1
vote
0 answers

Is the average Shannon's entropy of two groups of variables the mean of their two respective entropy values?

I have two groups of variables that respectively have Shannon's entropy of value X and Y. Does it make sense to consider the average entropy of those two groups to be the mean between X and Y?
iLikeKFC
  • 11
  • 2
1
vote
0 answers

I want to prove that multivariate normal distribution maximizes the entropy

I was going through probability higher than I am being taught I was trying this problem , Show that the multivariate normal distribution maximizes the entropy over all distributions with the same covariance matrix. Can anyone tell me how to solve…
Pulkit Joshi
  • 63
1
vote
1 answer

Is entropy "interpolative" (in the sense defined below...)?

Define the usual Shannon $H(p)=-\sum_i p_i\log_2(p_i)$ for discrete pdf (probability distribution function) $p=(p_i,i=1\ldots n)$. Now consider three pdf's: $p_a,p_b$ are given (with the same cardinality $n$), and $p_c$ is a convex combination…
John Forkosh
  • 712
1
2
3 4 5 6