0

I know this sounds like something I can just Google, but surprisingly, I cannot find it ANYWHERE. All I found is a document that says: "We will skip the variance of categorical distribution here." Source: https://livebook.manning.com/book/math-and-architectures-of-deep-learning/chapter-5/v-5/

I know Categorical Distribution is a generalization of the Bernoulli distribution (variance: $p(1−p)$). So what's the variance of Categorical Distribution? $p_i(1-p_i)$?

Thank you

Distribution Definition: https://en.wikipedia.org/wiki/Categorical_distribution

A_for_ Abacus
  • 1,899

1 Answers1

3

The definition of the variance of a random variable $X$ is (when it exists) $$ \operatorname{Var}[X] = \mathbb{E}[(X-\mathbb{E}[X])^2] =\mathbb{E}[X^2]-\mathbb{E}[X]^2 $$ Here, this is the same: if $X$ is distributed according to a categorical distribution $p$, we have $$ \operatorname{Var}[X] = \sum_{i=1}^k \left(i - \sum_{j=1}^k j p_j\right)^2 p_i = \sum_{i=1}^k i^2 p_i - \left(\sum_{i=1}^k i p_i\right)^2 $$ There is no general closed form for this.

Clement C.
  • 67,323