1

Let's say I repeated some trial 90 times and got a success on my last attempt.

I guessing that such a confidence interval is different than if I had gotten 1 success at any point out of my 90 trials.

In my case I didn't decide on 90 trials beforehand, but I'm stopping at 90 because I got my success.

How do I find a confidence interval in this case?

Simon
  • 31
  • 1
    What's the question? The desired confidence interval is supplied by the observer. You can't determine the answer for certain but you can, say, argue that the observed outcome has probability $<x%$ for some threshold $x$ that you supply. – lulu Feb 01 '23 at 18:12
  • It would be interesting to know what you think the two confidence intervals might be. For example R's binom package gives $11$ different methods for the binomial proportion confidence interval (your second paragraph) with noticeably different results. – Henry Feb 01 '23 at 18:36
  • @Henry If I got 1 success out of 90 I would guess the 95% confidence interval would be: .

    [1/90-1.96sigma ; 1/90+1.96sigma].

    Is this not the standard way to calculate a confidence interval?

     – Simon Feb 01 '23 at 19:14
  • That looks like a Wald interval. If you take $\sigma= \sqrt{ \hat p (1- \hat p)}$ with $\hat p = \frac1{90}$ I think you get an interval of about $[-0.19, 0.22]$ centred on $0.0111$. That looks implausibly wide to me and I may have made an error; but it is not much of a surprise as I would not expect the normal distribution to be a good approximation in such an extreme case. – Henry Feb 01 '23 at 19:41
  • I've used $\sigma =\sqrt{\frac{p*(1-p)}{n}}$ , but that still gives me an interval with negative values. What would be a more appropriate distribution to use in this extreme case? – Simon Feb 01 '23 at 19:46
  • For the binomial case, Wikipedia describes some possibilities – Henry Feb 01 '23 at 19:48
  • 1
    My experiments suggests one possible confidence interval for $p$ in your geometric distribution is from $0.02531781 , \hat p$ to $\min(1, 3.688879 , \hat p)$ which in your example of $\hat p =\frac1{90}$ would be about $[0.00028, 0.041]$. This approach seems to be close to a $95%$ confidence interval when $p$ is very small, and not too bad when $p$ is not so small (though clearly it is conservative in the upper tail when $p$ is large) – Henry Feb 01 '23 at 19:49
  • Was your experiment "perform 90 trials and record the number of successes", or was it "perform trials until the first success, and record the number of trials"? The two have different distributions. – ConMan Feb 02 '23 at 00:09
  • @ConMan The original question recognises that point, and asks for the effect of that distinction on the confidence intervals for the probability of a success. – Henry Feb 02 '23 at 01:44
  • @ConMan , that's what I was thinking, but I have no idea what distribution I should use in the case where I perform trials until first success. Could you give an example of how that could look? – Simon Feb 02 '23 at 08:29
  • The distribution of Bernoulli trials until first success is called the "geometric distribution". – ConMan Feb 02 '23 at 23:38

0 Answers0