1

I'm trying to gain a better intuitive understanding of the expected number of failures before the first success.

My current understanding is that in a geometric distribution where the probability of success is p, the expected number of trials up to and including the first success is 1/p.

So, for a biased coin with the probability of heads = 1/10, we would expect that it would take 10 flips on average before seeing a heads. This sort of makes sense intuitively. My question is: why is the expected value 10?

If X=# of trials up to and including the first success, then the expected value E(X) = SUM(x*P(x)) from x=1 to infinity. Would an intuitive understanding of this be something along the lines of: For each possible X, we calculate the probability for x and multiplied it against x and summed it up these values? I think of this process of being analogous the way we would calculate expected value when rolling a 6 sided dice i.e.: 

1/6*1 + 1/6*2 + 1/6*3 + 1/6*4 + 1/6*5 + 1/6*6

The only difference being in this case we have infinite possible values for X (whereas with a die there are only 6 possible outcomes). Is this the correct way to reason about the expected value in a geometric distribution?

Additionally, given that

P(X=1) = 1/10
P(X=2) = 9/10 * 1/10
P(X=3) = (9/10)^2 * 1/10
...
P(X=k) = (9/10)^(k-1)*1/10

Isn't the probability of taking at most 10 trials until a heads the summation of P(X=k) from k=1 to k=10 since they're mutually exclusive events? i.e. P(X=1) + P(X=2)...P(X=10) = ~0.65 (Wolfram Alpha Calculation)

How is this probability correlated with the expected value of 10? Or am I mixing apples and oranges here?

Anonymous
  • 113
  • 1
    What you describe (with probability of "success" $p$ in any single trial) is usually called a binomial distribution. Multiple repetitions are often called Bernoulli trials. The Wikipedia article on geometric distributions may help clear up the specific use of that term (authors typically use it for two related but different notions). – hardmath Jul 03 '18 at 23:54
  • 1
    One trick which helps in evaluating the expected value in this case is: if $X$ is a random variable with values in $\mathbb{N}$ then $E(X) = \sum_{n=1}^\infty p(X \ge n)$. – Daniel Schepler Jul 04 '18 at 00:00

4 Answers4

4

For an “intuitive” understanding of this result, I would look at it a different way. If you make a large number of coin flips, about 1/10 of them will come up heads. The expected value of a random variable is a type of average. Averages smooth out differences, so if you take that long sequence of coin flips and smear it out so that the heads are evenly spaced, they will come up every ten tries or so.

amd
  • 53,693
  • Great answer, but I still have a doubt. I'll first rephrase your answer in my own words, to see if I understood. Say we toss a biased coin, where in each toss we have a $p=\frac{1}{10}$ chance of landing heads. If we were to toss the coin 1000 times, we’d expect about 1 in every 10 tosses to be heads. So, we’d expect around 100 heads. They don’t necessarily have to be evenly spaced, but on average, if we toss enough coins, the heads will be about 10 tosses apart. So far so good... – joshuaronis Jun 30 '19 at 19:38
  • Here's the thing...if each heads is about 10 tosses from the previous heads, and we were to pick a random spot in the sequence to start at, wouldn't the expected value of tosses be 5.5, not 10? – joshuaronis Jun 30 '19 at 19:40
  • Actually, my previous comment doesn't make sense, since it's assuming that heads can at most be 10 tosses apart, when in reality, they can be more. I'm still a bit confused though... – joshuaronis Jun 30 '19 at 19:41
  • 1
    @joshuaronis : your confusion is the textbook example of the waiting-time paradox https://stats.stackexchange.com/questions/122722/please-explain-the-waiting-paradox – justt Apr 25 '22 at 23:14
3

Imagine rolling a die until a 1 is rolled, so $p=1/6$. Now roll this die $N=6000$ times -- you'd expect 1000 1's among those rolls. Mark all the 1's and measure the gaps between each pair of consecutive 1's in your sequence. Ignoring some fudging at the end (which goes to $0$ as $N$ goes to infinity) , the sum of the gaps is going be 6000, so the average gap will be 6000/1000 = 6. But the gaps are just values of a geometric random variable with $p=1/6$ so you can see why the expected value is 6, i.e. since it is the average value in repeated independent trials.

Ned
  • 3,852
2

If X is a geometric random variable with $$P(X=k) =q^{k-1}p $$ where $ q = 1-p $ we have $$E(X) = \sum_{k=1}^{\infty} kpq^{k-1} = p \sum_{k=1}^{\infty} k q^{k-1} $$

The trick is $ kq^{k-1} = \frac{d}{dq}q^{k} $

$$ E(X) = p \frac{d}{dq} \sum_{k=1}^{\infty} q^{k} = p \frac{d}{dq} \frac{q}{1-q}$$ $$= \frac{p}{(1-q)^{2}}= \frac{1}{p} $$

1

Remember, the expected value is really the probability-weighted average of all possible outcomes. You are right, if $X \sim \mathcal{G}(1/10)$, then $$\mathbb{P}[X \le 10] \approx 65\%,$$ however, that only accounts for the weights to the left, without accounting for weights larger than $10$ (and there is an infinite amount of points in those weights, although they have a finite weighted average).

gt6989b
  • 54,422