Frequency from probability

Question

Assume an event has a probability $p=1/100$ of happening, per trial. Here are three statements that I believe are true:

1. On average, the event is witnessed in a one-hundredth of the the trials.
2. The average number of trials between consecutive events is $100$.
3. It is more likeley than not to witness at least one event in a set of $69$ trials. This is found by computing the probability of the complement, i.e. the probability of no event in $69$ trials, which is $(1-p)^{69}=0.4998...$.

I can't understand how statements 2 and 3 can both be true. Is there a flaw in the reasoning somewhere ? Can they both be true? What is the "frequency" of the event ?

Answer 1

Let $X$ be the number of trials before one event (without loss of generality, we can look at the first time an event happens, i.e. start at trial $0$). Then, the second item can be rewritten $$ \mathbb{E}[X] = 100 $$ and if I am not mistaken, the third item is equivalent to saying that $$\mathbb{P}\{X > 69\} < \frac{1}{2}.$$ To see why this is not contradictory, you can for instance look at the formula for expectation $$ \mathbb{E}[X] = \sum_{n=1}^\infty n\mathbb{P}\{X=n\} = \sum_{n=0}^\infty \mathbb{P}\{X > n\}.$$

Looking at the second expression, the knowledge that $\mathbb{P}\{X > 69\} < \frac{1}{2}$ may lead to think that "only the first $69$ or so terms of the sum matter (after, the terms must be very small, as they are decreasing), so that the sum cannot be much more that $69\cdot 1 = 69$." However, this is not quite true: indeed, the terms of the sum after $n=69$ will get smaller and smaller, but you have infinitely many of them, and summing them together will add a significant contribution to the sum.

This is not very intuitive, but said differently (and in a very handwavy way): look at the first expression for the expectation, $\sum_{n=1}^\infty n\mathbb{P}\{X=n\}$. Item 3 means/hints that for $n > 69$, the value $\mathbb{P}\{X=n\}$ will be small (and actually getting smaller and smaller very quickly). But you multiply this quantity by $n$, which makes the term slightly bigger; and you sum all these things. So even if the probability that you have to wait, say, $3000$ trials, is very very small, it eventually adds up (along with the probability you have to wait $10^6$ trials, even smaller, etc), biasing the expectation towards higher values than you would "expect."

Answer 2

To add onto Clement's answer: If you think of the events as being the possible decay of a radioactive atom, with one trial per second, then your statements 2 and 3 reflect the difference between the mean life of the element (which would be about $100$ seconds), and the half-life of the element (which would be about $69$ seconds).

The inter-event time follows a geometric distribution, with the time $\tau$ between events having the probability distribution

$$ P(\tau = n) = p(1-p)^n $$

with $p = 1/100$. (You might write $n-1$ in the exponent, depending on what you mean by "between.") The continuous analogue of this is the exponential distribution, where the probability density function (PDF) of the inter-event time is given by

$$ f_\tau(t) = \lambda e^{-\lambda t} $$

with $\lambda = 1/100$. Here, $\lambda$ gives (as $p$ did, above) the event "rate," which you may notice is the derivative $f'_\tau(0)$. It turns out that for this distribution, the mean lifetime of an atom is given by $1/\lambda = 100$.

However, the half-life of the element is given by the cumulative distribution function (CDF), which is the definite integral, from $0$ to $t$, of the PDF. That is,

$$ F_\tau(t) = \int_{x=0}^t f_\tau(x) \, dx = 1-e^{-\lambda t} $$

The CDF gives the probability that an event happens before time $t$; equivalently, for this scenario, it gives the proportion of the sample that has decayed by time $t$. Thus, the half-life is governed by

$$ e^{-\lambda t} = \frac{1}{2} $$ $$ \lambda t = \ln 2 \doteq 0.69315 $$

which is why you found the half-life and the mean life to be related in approximately a $69:100$ ratio.

Answer 3

You have not formulated statement 2 as a precise statement, i.e., as the result of a well-defined, unambiguously stated probability problem.

As for statement 3, you have implicitly assumed the outcome per trial to be independent across trials, yet nowhere is this assumption included in your assumptions. Statement 3 can be false if the outcomes per trial are not independent. The correctness of a precisely stated version of statement 2 can also be affected by independence or not of the outcomes per trial.

As for your paradox under your interpretation of statement 2 and your implicit independence assumption, the bottom line is that not all probability distributions are symmetric, and therefore the expected value of a random variable need not be equal to its median value. In particular, the waiting time between occurrences of an event, when the event's occurrence is independent across trials, is not symmetric.

Edit: Regarding statement 2, consider the following probability distribution, which is fully consistent with the original problem specification: With probability 1/100, the event occurs on every trial, and with probability 99/100 it never occurs. In such case, the expected number of trials between occurrences of the event, given that it occurs, is 1. The probability that the event ever occurs is 1/100.

There are various ways to make statement 2 precisely defined and unambiguous. One way would be to define it as the expected number of trials between occurrences of the event, given that the event occurs. There are other possibilities as well, such as defining the expected number of trials between occurrences of the event as equal to expected number of trials between occurrences of the event if the probability that the event ever occurs equals 1, and as equal to infinity if the probability that the event ever occurs is less than 1. In the example provided, the former definition results in the answer 1, whereas for the second definition, the result is infinity.