0

Here, $H$ means the entropy function. I understand that the symbol $\oplus$ means modulo $2$ addition. But I don't understand the significance of the entire expression.

Ali Caglayan
  • 5,726
sprajagopal
  • 622
  • 1
  • 5
  • 20

1 Answers1

0

It means entropy of $X \oplus\hat{X}$ given that you already know the value of $\hat{X}$.

We remove one source of entropy by assuming that we know $\hat{X}$ so the only remaining source of entropy in the expression $X \oplus \hat{X}$ is $X$.

I.e. $H(X\oplus \hat{X}|\hat{X}) = H(X)$

This property does not apply generally for all operators but plus modulo 2 is special since it preserves entropy. Otherwise you would get

$H(X \oplus \hat{X} | \hat{X}) \leq H(X)$

since you wouldn't know how much the operator reduces entropy.

In general, the entropy of $H(X [\text{operator}] \hat{X})$ can be calculated if you know the probability distribution of $X [\text{operator}] \hat{X}$. For example, if both $X$ and $\hat{X}$ are binary (0 or 1) and have the probabilities

$P(X=1)=P(X=0)=P(\hat{X}=1)=P(\hat{X}=0) = 0.5$

Then the probability distribution of $X\oplus \hat{X}$ is

$P(X\oplus \hat{X} = 1) = P(X=0 , \hat{X} = 1) + P(X=1, \hat{X}=0) = 0.25+0.25 = 0.5$ $P(X\oplus \hat{X} = 0) = P(X=1 , \hat{X} = 1) + P(X=0, \hat{X}=0) = 0.25+0.25 = 0.5$

Then we can treat $Y=X\oplus \hat{X}$ just like any random variable $Y$ and say

$P(Y = 0) = P(Y = 1) = 0.5 \Rightarrow H(Y) = 2$

If we had used another operator, e.g. $Z = X * \hat{X}$ then $P(Z = 1) = 0.25$ and $P(Z=0) = 0.75$ we could calculate the entropy of Z and get another (lower) value of $H(Z)$.

Johan Wikström
  • 417