1

I am trying to understand the result in this thread (Dependent Bernoulli trials), which says the number of parameters in a joint distribution of n dependent Bernoulli random variables is $2^n - 1$.

The argument goes like this: The most flexible structure is the one that assigns to all possible n binary vectors $(x_1,…,x_n)$ a probabilty $P[x_1=i_1,…,x_n=i_n]=p_{i_1,…,i_n}$

Thus, you have to specify $2^n−1$ parameters. But I don't see why that's true?

Sam
  • 1,013

1 Answers1

1

Here $i_1\in\{0,1\}$, $\ldots$, $i_n\in\{0,1\}$. Each variable takes two values, so we have $2^n$ possible vectors $(i_1,\ldots,i_n)$. Say, for $n=2$ we have four possible vectors $(0,0)$, $(0,1)$, $(1,0)$, $(1,1)$.

Every vector $(i_1,\ldots,i_n)$ correspond to a probability $p_{i_1,\ldots,i_n}$ which is a parameter of joint distribution. And total number of parameters is one less than $2^n$ since total sum of the probabilities equals to one, and therefore one of the probabilities is entirely determined by the others $2^n-1$.

NCh
  • 13,807
  • 4
  • 17
  • 33
  • So basically you have to produce a probability value for each possible combination of outcomes, with the only constraint that they all add up to 1. – Sam Jan 23 '20 at 17:20
  • 1
    Yes, you are right. – NCh Jan 23 '20 at 17:23