4

This is probably a more basic question than this site is used to, probably because I'm only 13, and as such I'd appreciate if you gave a more basic and simple explanation than the norm for this site.

I was reading a BBC News article this morning and I found myself questioning the three diagrams, in the 'Why are two tests better than one?' section.

You do a first test and obtain nine heads and one tail... The probability that the coin is fair given this outcome is about 8%, [and the probability] that it is biased, about 92%.

You do a second test, and this time you throw eight heads and two tails. Now the probability for a fair coin is about 16%, for a biased coin about 84%.

So the naive thought might be that you haven't gained any certainty from this second test. But if you think about it differently, what you've really done is throw the coin 20 times and get 17 heads and three tails." This means there's a probability of 98.5% that the coin is biased.

I have three questions, all roughly along the same lines: Please see below paragraph, these questions are no longer 'active' - Since when is 9/10 heads a 92% probability, and why? - Why is 8/10 an 84% probability, I always thought that 8/10 = 80% - And finally, 17/20 is 85%, except when it's 98.5%. Why?

After some help from the comments, I now realise that it's not talking about the probability of getting a head (or a tail) but the probability of the coin being biased. Can someone explain (preferably in layman's terms) how the article gets to 92%, 84% and 98.5% respectively?

Thanks.

  • Take a look here: http://en.wikipedia.org/wiki/Checking_whether_a_coin_is_fair –  Apr 28 '13 at 08:28
  • These sentences from this article make no sense for me. Am I right or is it due to my difficulties with English ? – Stéphane Laurent Apr 28 '13 at 08:28
  • @Shahab Have you really read the sentences ? How to evaluate "the probability that the dice is not biased" ? Is there an underlying Bayesian approach used to evaluate $\Pr(p=0.5)$ ? – Stéphane Laurent Apr 28 '13 at 08:31
  • @StéphaneLaurent: The problem has many approaches. I believe the link is helpful. –  Apr 28 '13 at 08:36
  • @Shahab I had a brief scan, but I cannot see how it helps in my case. I can't understand half of the explanations, and for the ones I do partly understand I see no connection to my query. Thanks though. – Luke Ashford Apr 28 '13 at 08:38
  • @Shahab I'm statistician. If I well understand the problem, the only way to derive a probability $\Pr(p=0.5)$ that a dice is not biased from the results of dice throwings is a Bayesian-like approach. And there's no standard way to do that. – Stéphane Laurent Apr 28 '13 at 08:40
  • There is an example saying that 7 heads out of 10 indicates means roughly 13% chance (appropriately interpreted) of an unbiased coin. I thought this was similar to your question. Anyway, no issues. –  Apr 28 '13 at 08:40
  • @Luke: the connection is very strong indeed. In fact, the problem of your interpretation is that in the paper you cite, the problem is not to evaluate the probability of obtaining head (the result of the first test is then 90% as you say) but the problem is to understand from the result of the tests whether the coin is fair or biased. The probability of 92% they compute is precisely the probability that the coin is biased, after seeing the result of the first test [continues] – Simone Apr 28 '13 at 08:41
  • @Shahab No, the example you talk about is the probability $\Pr(0.45\leq p \leq 0.55)$. – Stéphane Laurent Apr 28 '13 at 08:42
  • Luke, are you really thinking that if 5/10 outcomes would have been heads would mean that there is a 50% chance that the coin is biased? Or to extrapolate even further, if only 1/10 had been heads, there is only 10% chance of the coin being biased? – Jyrki Lahtonen Apr 28 '13 at 08:44
  • @StéphaneLaurent: Yes, and that is why I use the word "roughly". You can make it more precise $Pr(0.49\le p\le 0.51)$, by integrating over those limits. –  Apr 28 '13 at 08:45
  • the link that shahab sent you explains a few methods of computing such probabilities. Stephane is suggesting that probably there exist better methods. Anyway, what you should first understand is the question they are trying to answer.

    You assume that a fair coin is a coin which gives you "head" (as well as "tail") with probability 50% (half of the cases). Your first test gives you "head" in 90% of the cases, so the probability that your coin is biased seems to be quite high (they say 92%) but there is no reason for this probability to be 9/10 as you suggest. I hope this helps you.

    – Simone Apr 28 '13 at 08:45
  • @Shabab and you will get $\Pr(p=0.5) =0$ ! The only way to get a positive probability is to use a Bayesian approach with a prior distribution on $p$ having a point mass at $0.5$. – Stéphane Laurent Apr 28 '13 at 08:46
  • @Simone Ahh, I see now. I'd mis-read the article. Can you, or anyone else (I'll edit the question) explain, in layman's terms, why it's 92%? – Luke Ashford Apr 28 '13 at 08:47
  • OTOH, if you get 6000 heads out of 10000 flips it is a moral certainty that the coin is biased. – Jyrki Lahtonen Apr 28 '13 at 08:47
  • @Simone I'm not trying to suggest better methods. I'm trying to understand where do the probabilities (92%,...) given by the article come from. That does not sound sensible. – Stéphane Laurent Apr 28 '13 at 08:48
  • @LukeAshford This is precisely what I was trying to understand. IMHO there's no standard solution to this problem. – Stéphane Laurent Apr 28 '13 at 08:49
  • @StéphaneLaurent: Yes, but if you define unbiasedness by $(0.49\le p\le 0.51)$ which (imho) is not wholly unreasonable, then you can get a rough probability of unbiasedness of the coin. If you do not wish to define unbiasedness this way, then of course this approach is not useful. –  Apr 28 '13 at 08:51
  • @Shahab Sure but there's no consensual definition. – Stéphane Laurent Apr 28 '13 at 08:53
  • I know that it is inaccurate for a small number of flips, but is it not a relatively useful quick'n'dirty method to calculate how many SDs you are away from the expected value (assuming an unbiased coin), and then read a percentage figure from normal distribution tables? It does feel like this gives a wrong kind of a conditional probability, but is it far off? – Jyrki Lahtonen Apr 28 '13 at 09:28

2 Answers2

6

Both you and the BBC news author suffer from a misunderstanding.

If a coin produces nine heads and one tail in ten tossings, it is reasonable to extrapolate and assume that even for $n\gg 10$ we may expect about $\frac9{10}n$ heads and $\frac 1{10}n$ tails. That is: By observing a random experiment (whatever that is) often enough we estimate an underlying probability that allows us to predict future repetitions of this experiment. This is where you get your $\frac 9{10}$ or $90\,\%$ from.

If a fair coin (i.e. one where heads and tails occur with probability $\frac12$ each) is tossed $n$ times, then the probability of exactly $k$ heads in this sequence is $\frac{n\choose k}{2^n}$. Expecially, with $n=10$ a sequence with exactly $9$ heads will occur with a probability of only $\approx0.0098%$. Actually, even an outcome of exactly five heads is not very common as it occurs with only probability $\approx 0.246$. So we should rather ask: What is the probability of an overdose of either heads or tails as extreme as obserevd or even more extreme? That is: What is the probability that nine or more heads or nine or more tails occur? This probability turns out to be $\approx0.021$. That is: in about $98\,\%$ of all experiments where you perform ten tosses with a coin known to be fair, you will observe that one of the two outcomes is extremely underrepresented (occurs at most once). This does not say, however, that the coin is not fair (after all I specified that the coin is known to be fair). This is also like the second part of the article says, that among sufficiently many nurses, even if they are "fair coins", a certain small percentage will trigger a test that is doomed to trigger a small percentage.

What the BBC article (or rather Colmez) claims about the coins, however, is that the experiment shows that the probability that the coin is biased is $92\,\%$. This is something that cannot be concluded in this extend in any way.

Assume you have a big bag of $N$ coins. You know that one of these coins is biased - it produces heads with probability $p>\frac12$ instead of $\frac12$. You draw a coin at random. What is the probability that it is faked? Obviously, without further tests, it is $P(B)=\frac1N$ (where $B$ denotes the event that the coin is biased, so $\overline B$ is the event that the coin is fair). Now you perform our test, ten tosses, and event $A$ "nine heads" occurs. What do we learn from that? For a fair coin, $A$ occurs with $P(A|\overline B)={10\choose 9}\cdot 2^{-10}$, for a fake coin, $A$ occurs with $P(A|B)={10\choose 9}\cdot p^9(1-p)$. As we don't know which coin we used, we know beforehand only that $$P(A)=P(\overline B)P(A|\overline B)+P(B)P(A|B)=\left(1-\frac1N\right){10\choose 9}\cdot 2^{-10}+\frac1N{10\choose 9}\cdot p^9(1-p)$$ Now Bayes theory comes in and says that $$ P(B|A) =\frac{P(B\land A)}{P(A)} = \frac{\frac1N{10\choose 9}\cdot p^9(1-p)}{\left(1-\frac1N\right){10\choose 9}\cdot 2^{-10}+\frac1N{10\choose 9}\cdot p^9(1-p)} = \frac{ p^9(1-p)}{\left(N-1\right) 2^{-10}+ p^9(1-p)}.$$ Well, obviously this expression depends on both $N$ and $p$ and can hardly be equated to $92\,\%$ just like this. For example if we had $p<\frac12$, then the observation of $A$ would even lower the probability of the coin being faked. Or if the fake coin has heads on both sides ($p=1$) it is even impossible that the coin is the faked one. Even in an "optimal" situation where the coin has $p=\frac9{10}$ (so that nine heads, one tails would be the most likely outcome for that coin), we find $$ P(B|A) =\frac{387420489}{9765625N + 377654864}. $$ For big $N$ this is $$P(B|A)\approx39.7\cdot \frac1N,$$ so this test can hardly tell us that the coin we have in our hands is the biased one with probability $92\,\%$ if $N$ is sufficiently big! With $N=1000$ coins in our bag, we could raise our confidence from one permille to about four percent, way less than $92\,\%$. In fact, if you only have four coins ($N=4$) and one of them is biased such that it makes the test most predictive (i.e. the one biased coin has $p=\frac9{10}$), then the test (if it results in nine heads as outcome!) allows you so say that with probability $93,\%$ the coin in your hand is the heavily biased one (as opposed to that you could only make the same claim with $25\,\%$ confidence before the test).

Nevertheless, the main point is correct: If you perform several independent(!) tests with low reliability, then collectively these can give high reliability. A single photon hititng your retina cannot tell you anything and yet with a few gazillion photons you recognize exactly what goes on in your environment.

  • That hit the nail right on the head. I wont lie, some bits went in one ear and out the other but I got the meaning of everything, and your last paragraph rounds it off nicely. As soon as I get enough rep I'll upvote. – Luke Ashford Apr 28 '13 at 10:33
2

the definition you use for the probabilities is $\frac{N_{cases}}{N_{total-cases}} $, which is a definition that applies in certain circumstances. But the way probablities are computed in this problem are different. They involve a more complex problem that has a more complex response. The question is, what is the probability of getting 9 heads an one tail, if the coin is not biased, and what is the probablity that a casee like that is the result of a bias. They involve a more advanced study of probablility theory beyond the definition that you use.

  • Yeah, I understand that it's not the simple question I thought it was. Can you explain how/why it's 92% chance of bias? – Luke Ashford Apr 28 '13 at 09:04
  • @StéphaneLaurent I think it means that there's 98.5% probability that p != 0.5 after 17 heads. I could be wrong though, I mis-understood before. – Luke Ashford Apr 28 '13 at 09:10
  • yes, that what it means, but please give me until tomorrow to try to remember how to calculate that. Not that it is difficult, but it is easy to loose your practice – Wolphram jonny Apr 28 '13 at 09:12