0

I've been thinking about a basic question recently. It is well known that the Bernoulli distribution $$p_k = {N \choose k} p^k (1-p)^{N-k} \tag{1} $$ asymptotes

  • (i) to a Gaussian distribution if $p=const$ and $N\to\infty$ (also $1\ll k \ll N$) and
  • (ii) to a Poisson distribution if $p\to 0$ and $N\to \infty$ precisely such that $pN=\lambda=const$.

One possible proof of $(i)$ makes use of Stirling's approximation to simplify the factorials in the binomial factor. Here, I am more interested in (ii). One way to show (ii) is simply to write out the factorials in the binomial coefficient as products, cancel factors and then use the well-known formula $(1+x/N)^N\to e^x$ as $N\to \infty$. I understand that derivation.

Nonetheless, as in both (i) and (ii) $N\to \infty$, it seems tempting to use Stirling's approximation again, but somehow when I try to do so, I do not manage to get Poisson's distribution back out. Here's what I tried:

Use Stirling's approximation in the form $n! \sim n^ne^{-n}$ (where $A \sim B$ means "$A$ asymptotic to $B$") such that $\frac{N!}{(N-k)!}\sim \frac{N^N}{(N-k)^{N-k}} e^{-k}$. Then

$$p_k \sim \frac{N^N}{(N-k)^{N-k}} e^{-k} \frac{\lambda^k}{N^k} \frac{\left(1-\frac{\lambda}{N}\right)^{N-k}}{k!}$$ Next, simply rearrange and use $(1-k/N)^N \sim e^{-k}$ to find

$$ p_k \sim \left(1-\frac{k}{N}\right)^k \left(1-\frac{\lambda}{N}\right)^{N-k} \frac{\lambda^k}{k!} $$

Now, comparing to the Poisson distribution, I would expect that the first two terms are asymptotic to $e^{-\lambda}$. However, if I plug in large numbers for $N$ and $k$, this is not the case. What is the problem?

Community
  • 1
Cyclone
  • 1,853
  • 1
    FYI that isn't the Bernoulli distribution. That's the Binomial distribution, which is a sum of N Bernoulli distributed random variables. – kevinkayaks Dec 22 '18 at 03:43

1 Answers1

1

I have found the problem. In fact, the first two terms converge to $e^{-\lambda}$, but only under the condition that $k^2\ll N$. In that case:

$$ \left(1-\frac{k}{N}\right)^k \left( 1-\frac{\lambda}{N}\right)^{N-k} \sim \underbrace{\left( 1- \frac{k^2}{N} + \mathcal{O}\left(\frac{k^4}{N^2}\right) \right)}_{\sim \, 1} \underbrace{\left( 1-\frac{\lambda}{N}\right)^{-k}}_{\sim\, 1} \underbrace{\left( 1-\frac{\lambda}{N}\right)^N }_{\sim \, e^{-\lambda}} \sim e^{-\lambda} $$

When choosing $k$ and $N$, I had at first not observed this requirement.

Cyclone
  • 1,853