How to calculate the sum of that probability distribution?

Question

Let $W$ be the random variable that counts the number of tails before one gets $r$ heads for a coin whose probability of heads is $θ$. Without using moment generating function, show that the mean and variance for $W$ are $[r(1-θ)]/θ$ and $[r(1-θ)]/θ^2$. I know it is a negative binomial distribution. But without using mgf, how to calculate that? I tried and I gave up in the half way because it is too complicated.

Answer 1

$W$ is the sum of $r$ independent random variables $W_1,\ldots,W_r$ each of which counts the number of tails before one gets one head. Therefore the mean and variance are respectively $r$ times the mean and variance of any of those independent random variables.

So $\Pr(W_1=k)=(1-\theta)^k\theta$ for $k=0,1,2,3,\ldots\,{}$. One way to find $\operatorname{E}(W_1)$ is just applying the definition: \begin{align} \operatorname{E}(W_1) & = \sum_{k=0}^\infty k\Pr(W=k) = \sum_{k=0}^\infty k(1-\theta)^k\theta \\ & = \sum_{k=0}^\infty \sum_{j=1}^k (1-\theta)^k \theta \quad (\text{This last sum has $0$ terms when $k=0$.}) \\ & = \sum_{j=1}^\infty \sum_{k=j}^\infty (1-\theta)^k \theta \\ & = \sum_{j=1}^\infty \frac{(1-\theta)^j\theta}{1-(1-\theta)} = \sum_{j=1}^\infty (1-\theta)^j = \frac {1-\theta} \theta. \end{align}

To find $\operatorname{E}(W_1^2-W_1)$ is algebraically a bit simpler than to find $\operatorname{E}(W_1^2)$ directcly: \begin{align} \operatorname{E}(W_1^2-W_1) & = \sum_{k=0}^\infty k(k-1)(1-\theta)^k\theta \\ & = \theta(1-\theta)^2\sum_{k=2}^\infty k(k-1)(1-\theta)^{k-2} \\ & = \theta(1-\theta)^2\sum_{k=2}^\infty k(k-1)\eta^{k-2} \\ & = \theta(1-\theta)^2\sum_{k=2}^\infty \frac{d^2}{d\eta^2} k\eta^{k-2} \\ & = \theta(1-\theta)^2 \frac{d^2}{d\eta^2} \sum_{k=2}^\infty \eta^k \\ & = \cdots \end{align} The last series is geoemtric. Add it up; do the differentiation; then replace $\eta$ with $1-\theta$. Then by using the known value of $\operatorname{E}(W_1)$, you can find $\operatorname{E}(W_1^2)$ and then $\operatorname{var}(W_1)=\operatorname{E}(W_1^2)-(\operatorname{E}(W_1))^2$.

Two issues:

• That $\sum\dfrac{d}{d\eta}\cdots=\dfrac{d}{d\eta}\sum\cdots$ is easily shown when the sum has finitely many terms. That it works with infinitely many terms is problematic. A well-known theorem that's not that hard to prove says it does work when the sum is a power series and one is in the interior of the circle of convergence.

• Perhaps it is inelegant for me to use two very different methods for the two expected values. I have a preference (on this particular occasion of this particular posted question) for the first method because it avoids the issue in the first bullet point above. (The second method might be somewhat more generally applicable and thus preferable in some other contexts.) My excuse is that there are only 24 hours in a day and I must attend to some other things.