0

The sum I would like to minimise is

$$f(a)= \sum^n_i a_i\log_2 a_i $$

on the constraint that $a_i \in (0,1]$. If I take the gradient of this function with respect to each $a_i$ I obtain $$a_i=e^{-1}.$$

This problem stems from not being able to understand a problem in a textbook of mine, which says that

$$g(a) = \sum^n_i a_i\log_2 (1/a_i)$$

is maximised when all $a_i$ are equal (under the constraint that a_i represent probabilities). Am i correct in saying that maximising $g$ is the same as minimising $f$?

  • If $n=1$, $f(a)=a\log_2(a)$, $f\left( e^{-1}\right)= -0.5307$ and $f(1)=0$. ($\frac 1e$ is certainly the minimum in that case). – lulu Jun 15 '19 at 10:56
  • How does $a_i = 1/n$ give something smaller? Since $n$ doesn't appear in the terms I would think that whatever value of $a$ minimizes $a$ log_2 $a$ would be the value for all $a_i$ to minimize the sum. – lurker Jun 15 '19 at 11:05
  • I have edited my post –  Jun 15 '19 at 11:18
  • $g(a)=-f(a)$ so maximizing $g$ is the same as minimizing $f$, not $-f$. – lulu Jun 15 '19 at 11:29
  • yes you are correct. But still, why is my textbook making this claim that a_i = 1/n is the solution? –  Jun 15 '19 at 11:41
  • 1
    Is your textbook perhaps adding another condition, such as $a_1+a_2+\cdots+a_n=1$? – Barry Cipra Jun 15 '19 at 11:43
  • @BarryCipra I also think that: it is well known that the entropy of the uniform probability is maximum, otherwise consider $a_1 = \ldots = a_n = 1$... – dcolazin Jun 15 '19 at 11:51

0 Answers0