How confident am I to say the coin has been rigged

Question

Let's just say that I flipped a coin 100 times, got 80 heads and 20 tails, how confident am I to say that the coin has been rigged? Which statistical test should I use?

thanks!

Answer 1

This is a binomial distribution. Check, e.g., this tutorial on how to formulate the null hypothesis ($p=0.5$) and what is required to reject it at a significance level $\alpha$.

Answer 2

The probability that the coin is rigged will depend on your priors. If you hadn't flipped the coin a bunch of times, how likely would you have said that it was rigged? The answer depends on various things: coins in math textbook thought experiments rarely are rigged. Coins that friends provide in order to bet against you might be rigged a lot, at least if your friends are like some of mine. Rosencrantz's coin is rigged by the text of Stoppard's play.

Once you have the priors in place, you can update your probabilities using Bayesian reasoning. For simplicity, I will assume that every coin is either rigged or fair, and with no other knowledge a coin is rigged with probability $P(\text{rigged})$. Then there's some pretty small probability $P(\text{80/100 heads}|\text{fair})$ of a fair coin showing this data, and some much higher probability $P(\text{80/100 heads}|\text{rigged})$, which depends on exactly what version of "rigged" you are imagining.

Then before running the experiment, you might imagine that the universe is in one of the following four possibilities, with the following probabilities:

$$ \begin{array}{c|c|c|} & \text{fair} & \text{rigged} \\ \hline \text{80/100 heads} & P(\text{80/100 heads}|\text{fair}) (1 - P(\text{rigged})) & P(\text{80/100 heads}|\text{rigged}) P(\text{rigged}) \\ \hline \neg(\text{80/100 heads}) & (1-P(\text{80/100 heads}|\text{fair})) (1 - P(\text{rigged})) & (1 - P(\text{80/100 heads}|\text{rigged})) P(\text{rigged}) \\ \hline \end{array} $$

With the new information, you know that you're in the top row. The assumption is that the relative likelihood of those events hasn't been updated, so that

$$ \frac{P(\text{rigged}|\text{80/100 heads})}{1 - P(\text{rigged}|\text{80/100 heads})} = \frac{P(\text{80/100 heads}|\text{rigged}) P(\text{rigged})}{P(\text{80/100 heads}|\text{fair}) (1 - P(\text{rigged}))} $$

or, solving for $P(\text{rigged}|\text{80/100 heads})$, we get:

$$ P(\text{rigged}|\text{80/100 heads}) = \left( 1 + \frac{P(\text{80/100 heads}|\text{fair}) (1 - P(\text{rigged}))}{P(\text{80/100 heads}|\text{rigged}) P(\text{rigged})}\right)^{-1} $$

Let's now make some simplifying estimates. The error will probably be absorbed by not really having a good guess for the priors of having a rigged coin. First, $P(\text{80/100 heads}|\text{fair})$ is really small, around $10^{-k}$ for $k$ between $10$ and $20$, much smaller than the denominator of the fraction on the right-hand side. So the right-hand side is roughly $$ P(\text{rigged}|\text{80/100 heads}) \approx 1 - \frac{P(\text{80/100 heads}|\text{fair}) (1 - P(\text{rigged}))}{P(\text{80/100 heads}|\text{rigged}) P(\text{rigged})} $$ Second, let me assume that $P(\text{80/100 heads}|\text{rigged})$ is pretty close to $1$, and round it to that.

So everything is left in $P(\text{rigged})$. What's your guess? I'd say that if you are pulling a random coin from circulation, the probability that it's rigged is in the $10^{-j}$ range for $j$ close to, but a little less than, $10$? And if it's a magic trick, then maybe $10^{-2}$? Either way, we can clearly round $1 - P(\text{rigged})$ to $1$, and we're left with: $$ P(\text{rigged}|\text{80/100 heads}) \approx 1 - \frac{P(\text{80/100 heads}|\text{fair})}{P(\text{rigged})} \approx 1 - 10^{j-k} $$

For my estimates, this is extremely close to $1$, since $k$ is a bit bigger than $j$.

But, if on the other hand you had only flipped the coin $10$ times, and seen $8$ heads, you could try the same analysis, with the difference that $P(\text{80/100 heads}|\text{fair}) \approx 10^{-2}$. Now, if I guess $P(\text{rigged})$ around $10^{-5}$, I cannot approximate $\left(1 + x\right)^{-1} \approx 1-x$. Instead, I estimate: $$ P(\text{rigged}|\text{80/100 heads}) \approx \left( 1 + \frac{P(\text{80/100 heads}|\text{fair})}{P(\text{rigged})}\right)^{-1} \approx \left( 1 + 10^{3} \right)^{-1} \approx 10^{-3} $$

So somewhere between flipping the coin $10$ times and $100$ times, $P(\text{rigged}|\text{results of experiment})$ switches dramatically from "almost $0$" to "almost $1$". Again, exactly when this occurs depends on the un-updated $P(\text{rigged})$. The change-over is when $P(\text{results of experiment}|\text{fair})$ and $P(\text{rigged})$ are at the same order of magnitude.

I should comment too that in these estimates, the exponential behavior of probabilities is totally swamping any fine effects, so that we really can round to some power of $10$, at least at a first pass of deciding when to change our beliefs based on further evidence.

Answer 3

If by "rigged" you mean "heads biased", then you will need to set up a 1 sided test with your null hypothesis being prob(head) = 1/2 and alternative hypothesis as p>1/2. You then need to work out the probability of achieving 80 heads and 20 tails OR WORSE (meaning more than 80 heads) with p=1/2 (the null hypothesis) and choose your significance level.

Looking at Doctor Dan's answer.. it would be advisable to pay special attention to the section on using the normal distribution to approximate the binomial distribution. Working out the above probability by hand isn't going to be quick.

Answer 4

If you aren't sure where the different answers come from, have a look at the following wikipedia pages:

http://en.wikipedia.org/wiki/Statistical_hypothesis_testing

http://en.wikipedia.org/wiki/Bayesian_inference

The approach you are most likely to be familiar with is the Neyman Pearson approach described by Doctor Dan and I, whereas Theo has used Bayesian inference which requires a conjecture about priors...

This would have been posted as a comment but I don't have enough rep.