2

Could someone please explain the logic of the reasoning behind why we reject the $H_0$ if the $\alpha$ exceeds the $p$ value

What I understand is that $\alpha$ is the probability of making a type 1 error if $H_0$ is true.

$p$ value is the likelihood of getting a value more to the right or left of your test stat if the $H_0$ is true (probability of getting a test stat that favors $H_a$ more)

So if $\alpha>p$ value means we reject $H_0$

Then it must mean that if the probability of making an error while $H_0$ is true is larger than the probability of getting a value more the to the right of your test statistic then reject $H_0$. How does that logic add up? Could someone explain the reason behind the why we reject $H_0$?

Thank you

Sebastian Nuxoll
  • 300
  • 1
  • 10
Ved Brahmbhatt
  • 65
  • 1
    I suggest you read the (short) section on p values in Shao's mathematical statistics. It's explained quite clearly – Andrew Sep 26 '23 at 00:45
  • Your test is reject the null hypothesis when the result appears too extreme on the assumption the null hypothesis is true. The alternative hypothesis helps you decide what counts as extreme. You decide $\alpha$ in advance and design the rejection region so $\alpha$ is the probability of seeing results in the rejection region on the assumption the null hypothesis is true, thus quantifying too. $p$ is the probability of seeing the result you did see or one as or more extreme on the assumption the null hypothesis is true, so $p <a$ suggests the result seen would be in the rejection region – Henry Sep 26 '23 at 15:23

1 Answers1

2

The idea behind hypothesis testing is that if a hypothesis is true, it is far more probable that our data will be in agreement with it than not. However, it won't always be such, since evidence can be misleading. We set $\alpha$ to control the chance that we'll be mislead when the hypothesis is true.

It is traditional to set $\alpha=0.05$ but that is just a tradition. For example, at CERN LHC (particle accelerator that found the Higgs) they use different $p-$value cutoffs so as to control false positives.

The $p-$value just tells us the smallest $\alpha$ we'd need to have to not reject.

Conversely, by definition the $p-$value statistic has a $\text{Uniform}(0,1)$ distribution, so it will fall below $\alpha$ exactly $\alpha$ percent of the time when $H_0$ is true.

Basically, its just another way of coming up with the rejection region. You can use case-specific cutoffs (e.g., T-statistic is greater than $t$) or convert to $p-$values which are uniformly interpretable.

Both approaches lead to the same thing, although I like $p-$values because they show the degree of disagreement.

Annika
  • 6,873
  • 1
  • 9
  • 20