Mathematics Stack Exchange
Mathematics Stack Exchange

Questions tagged [entropy]

This tag is for questions about mathematical entropy. If you have a question about thermodynamical entropy, visit Physics Stack Exchange or Chemistry Stack Exchange instead.

This tag is for questions about mathematical entropy, not to be confused with thermodynamical entropy which goes in Physics or Chemistry Stack Exchange.

1617 questions
0
votes
1 answer

How can $H(p, 1-p)$ be plotted?

So, I get of course that I can plot the entropy $- x \log_2(x)$ just as is which gives us: This is because we say that $p = (p_1, \ldots, p_n)$ and we have \begin{align*} H(p) = H(p_1, \ldots, p_n) &= - \sum_{k=1}^n p_k \log_2 p_k \end{align*} Now…
Stefan Falk
  • 1,217
  • 1
  • 11
  • 24
0
votes
2 answers

Derivation of joint entropy $H(X,Y) = H(X) + H(Y|X)$

Can somebody explain the calculations with arrows below? And I am sorry if I have placed my post in the wrong place.
user122358
  • 2,712
0
votes
1 answer

Estimate password entropy by "trying out" passwords

The entropy of a password of a fixed length $n$ and $c$ possible characters is calculated by $n*log_2(c)= log_2(c^n)$, see also here: https://ritcyberselfdefense.wordpress.com/2011/09/24/how-to-calculate-password-entropy/ Assuming that I have a…
mario.schlipf
  • 103
0
votes
1 answer

Joint entropy calculation of discrete random variables

Suppose that i want to calculate the joint entropy $H(A,B)$ of two discrete random variables of the form: $A=\{-1,1,1,-1,-1,-1,1,1\}$ and $B=\{1,-1,1,1,-1,-1,-1,1\}$. If the goal was just the calculation of the entropy of A or B, then, for example,…
john
  • 67
0
votes
0 answers

Why is $h(T)=\lim_{n\to\infty}\frac{1}{n}\log \#\mathcal{B}_n$?

I am reffering to this site: http://www.scholarpedia.org/article/Topological_entropy Definitionj of topological Entropy by Adler, Kohnheim For an open cover $\mathcal{U}$ of $X$, let $N(\mathcal{U})$ denote the smallest cardinality of a subcover of…
user34632
0
votes
1 answer

Entropy of $Y=bX$

If I have two random variables $Y$ and $U$ related as $Y=bU$, where $b>0$ is a constant and knowing that $\text{H}(x)$ represents the shannon entropy, such that: $$ \text{H}(x)=−\int \text{p}(x) \ \text{log}_2(\text{p}(x)) \ dx $$ Then, what is the…
Mauro
  • 313
0
votes
1 answer

Greenberg-Hastings-Model: What kind of shift space is it?

I would like to read something about the entropy of the one-dimensional Greenberg-Hastings-Model - and I think maybe I can find something about that in the book "Symbolic Dynamics and Coding" - but I do not know which kind of shift space the…
Salamo
  • 1,094
-1
votes
1 answer

Prove that: H(X, Y |Z) ≥ H(X|Z)

could someone help me how to solve this proof which is related to entropy? Should I subtract H(X|Z) from both sides ?
Jack
  • 1
-1
votes
1 answer

Stuck with math proof (proving maximum and minimum entropy)

I am trying to prove that max. entropy is when p = 1/2 and p = 1 for minimum. So this is the setup. I cannot understand how they were able to derive from original function to another function so quickly, as shown there. My calculus is rusty, but…
John Doe
  • 1
1 2 3 4 5
6