3

Good evening, I am a little confused about type I errors. If you are given a population of students doing hypothesis tests for a certain condition at a certain significance level, is it possible to calculate:

(a) how many students will fail to reject the null hypothesis given that the null hypothesis is false

(b) how many students will reject the null hypothesis given that the null hypothesis is true.

I have tried searching online but so far all sites only show how to calculate the probability that at least one type I error will be made. Any assistance will be greatly appreciated. Thanks!

  • What do you mean by "how many people will ..."? Who are these people and what are they doing? – Robert Israel Nov 07 '17 at 21:33
  • oh sorry for the vagueness, I'm paraphrasing my question. The people are doing hypothesis tests for a condition. In this example, the population refers to the people doing the hypothesis tests, the significance level refers to the level of significance that each person is testing at. – ImRellyBadAtMath Nov 07 '17 at 21:44
  • ok I have edited my post, hopefully it is clearer now what the question is – ImRellyBadAtMath Nov 07 '17 at 22:18
  • 1
    The significance level is suppose to be the probability that the null hypothesis will be rejected if it is true. – Henry Nov 07 '17 at 22:45

1 Answers1

2

This is easiest to think about if you have a 'simple' null hypothesis and a simple alternative. Suppose your data is summarized as a sample mean $\bar X$ of $n = 16$ independent observations from $\mathsf{Norm}(\mu, \sigma=4)$ and you are testing $H_0: \mu = 20$ vs. $H_1: \mu = 30.$

Then the test statistic is $$Z = \frac{\bar X - 20}{\sigma/\sqrt{n}} = \bar X - 20.$$ Then, under $H_0,$ we have $Z \sim \mathsf{Norm}(0,1).$ Testing at significance level $\alpha = 0.05 = 5\%,$ you would reject $H_0$ if $Z \ge 1.645.$ That is to say $P(\text{Reject} H_0\,|\,H_0 \text{True}) = P(\text{Type I Error}) = \alpha = 0.05.$ So if 100 students are performing this test and if $\mu = 20,$ then you would expect five of them to Reject even though $H_0$ is true. This is the answer to part (b).

In my contrived simple example, there is only one way for $H_0$ to be false and that is to have $\mu = 30.$ Then it is easy to find $$P(\text{Reject } H_0\,|\, H_0 \text{ False}) = P(\text{Reject } H_0\,|\, \mu = 30) = P(Z = \bar X - 20 \ge 1.645\,|\,\mu = 30).$$ That is the "power of the test" sometimes denoted $\gamma = 1 - P(\text{Tyee II Error}) = 1 - \beta.$ You could use $\beta$ to answer part (a).

In practical applications, the alternative hypothesis usually has many values: for example, $H_1: \mu > 20.$ Then the power and the probability of Type II Error are functions of the possible alternative values of $\mu,$ sometimes written as $\gamma(\mu_1)$ and $\beta(\mu_1),$ where $\mu_1 \in H_1.$ In such a situation, the power function and the Type II error function have to be evaluated one point at a time. Accordingly, there would be multiple answers to part (a).

BruceET
  • 51,500