Prove that: H(X, Y |Z) ≥ H(X|Z)

Question

could someone help me how to solve this proof which is related to entropy? Should I subtract H(X|Z) from both sides ?

score 1 · Answer 1 · answered Apr 17 '20 at 14:30

Note that

$H(X,Y|Z) - H(X|Z) = I(X,Y|Z)$

where $I(X,Y|Z)$ is the mutual information between $X$ and $Y$ given $Z$.

Now, the proof that the mutual information is always non-negative is simple, and involves Jensen inequality.

See

Cover, T.M.; Thomas, J.A. (1991). Elements of Information Theory (Wiley ed.). ISBN 978-0-471-24195-9.

1 Answers1