4

I got in a bit of an discussion with my dad about this and thought about it a lot. Would like some insight.

So the question: if you have a test (ex: a virus antibody test) that returns positive or negative and has a 30% chance of having an incorrect result, how many tests would you need to make to have significant confidence (>99%) that the overall mode of all tests is correct?

My dad argued that repeating tests does not increase confidence and I thought that it would.

Robert Cordingly
  • 41

2 Answers2

3

Your dad is wrong, assuming that the tests are independent. In this case, increasing the number of tests does actually increase confidence.

Let's make a couple of concrete examples, i.e., let's fix the number of tests and see what the probability of an incorrect result is.

  • One test:

This one is easy, the result is incorrect with probability $0.3$.

  • Two tests:

This one is tricky. We don't know how to break ties! Is a tied result correct or incorrect? OK, let's say in case of a tied result, we take another test. What are the chances our overall result will be incorrect?

Well, the probability that both results are incorrect is $0.3 \cdot 0.3 = 0.09$. The probability that both results are correct is $0.7\cdot 0.7 = 0.49$.

So, if you take two tests, you will get the correct answer $49\%$ of the time, an incorrect answer $9\%$ of the time, and an inconclusive answer $42\%$ of the time. But if the result is inconclusive, then the THIRD test will yield the correct result with a probability of $0.7$, meaning the $42\%$ of the time, $70\%$ of the time, the tie breaker will yield the correct result! Overall, $70\%$ of $42\%$ is $29.4\%$, which means that the total results are:

  • Correct result with no ties $49\%$ of the time.
  • Incorrect result with no ties $9\%$ of the time.
  • Tie, followed by the correct result $29.4\%$ of the time
  • Tie, followed by an incorrect result $12.6\%$ of the time.

Overall, you can see that you will eventually get the correct result $49\% + 29.4\% = 78.4\%$ of the time, so your confidence will increase!

In general, if tests are independent, then repeating them and counting correct results yields what is called the binomial distribution. Now, you are only counting examples when the mode is correct, which means that, for an odd number of repeats, say $n$, you are only asking when $$P(X=n)+P(X=n-1)+\cdots +P(x=\frac{+1}{2}) > 0.99$$

If you try plugging in a couple values of $n$, you should see that this happens at a not-too-high, but also not-that-low value of $n$. For the first $20$ odd values of $n$, the probabilities are as follows:

n  | P(more than half are correct)
---+-------------------------------
1  | 0.7
3  | 0.784
5  | 0.83692
7  | 0.873964
9  | 0.90119134
11 | 0.92177520904
13 | 0.9376247882008
15 | 0.9499874599462239
17 | 0.9597230639457454
19 | 0.967446643118699
21 | 0.973610059298716
23 | 0.978551998453966
25 | 0.9825302594739422
27 | 0.9857434702977692
29 | 0.9883461710650691
31 | 0.9904595640881166
33 | 0.9921793376606215
35 | 0.9935814589379696
37 | 0.9947265246478039
39 | 0.9956630678652156
5xum
  • 123,496
  • 6
  • 128
  • 204
2

It depends.

If consecutive tests are independent, then repeated testing can increase the confidence in the result.

If the tests are dependent, this need not happen. E.g., assume that in a population with same number of men and women, 70% of men have brown hair and 30% are blond, whereas 70% of women are blond and 30% have brown hair. Then you can "test" for gender by pulling out a hair and checking its colour and this test is 30% wrong. But if you repeat this test with the same person 1000 times, you will always get the same hair colour and hence either always the wrong or always the right result.

Hagen von Eitzen
  • 374,180