2

Please help me with this basic question on statistics: If a standard brick is dropped on a standard raw chicken egg from 1 meter; the egg breaks. How many times does this dropped-brick-onto-egg need to be repeated to establish statistical significance? At what point can it be stated that: "It has been proven that a brick dropped onto a raw egg will cause the egg to break" or "It has been proven to a 99% percent certainty."

(This is not a riddle or a joke. I'm trying to understand this basic concept.)

A J
  • 21
  • 1

2 Answers2

1

Laplace dealt with similar issues when he asked "What is the probability the sun will rise tomorrow?" It seems that there are things that we can know with certainty, which a priori doesn't seem to mesh with hypothesis testing.

However, note that before you can perform a hypothesis test, you need to understand what the world would look like under the null hypothesis. If your null hypothesis is: "A standard brick cannot break a standard egg when drooped from 1 meter", then all it takes is one demonstration to the contrary to be 100% sure. A more interesting case would be if you said there is a 99.999% chance of the brick breaking the egg (as a null hypothesis) vs the probability being greater than 99.999%. This would require a very large number of trials.

But...all this assumes that there is a random relationship between the brick hitting th egg and it breaking. At the macroscopic level, this apperars to be incorrect. A physicist or mechanical engineer, if given the exact parameters of your experiment, would be able to predict with almost certianty, what will happen. In fact, violoations of their predicitons would prompt an investigation of your expreiment, not skepticism of their calculations. So, if you have a cauasal law that is virtually certain to apply, then hypothesis testing is not an appropriate way to resolve this question....you need some actual randomness and some model of that randomness.

  • Thank you for your thoughtful reply. I think I have come closer to understanding some important elements within my question. I have done much reading on the web about this question and I now realize that I am actually asking about an observational study (OS) -- not an experiment. An OS does not need a null hypothesis or a random relationship. But what I still don't know is how to apply statistical analysis to the data collected in an OS. For example, if I observed and recorded the speed at which cars travel through an intersection, how many cars would I need to make the study valid? – A J Jul 19 '15 at 00:02
  • @AJ your study is valid if it adheres to its assumptions. Even in an observational study, you are attempting to answer some question, correct? This question can be formulated as a hypothesis test. Now, unless you have perfectly noiseless data, you will have some statistical error that you will need to characterize to ensure you do not mistake random errors with a significant finding. –  Jul 19 '15 at 00:30
  • Bey, this is where I get lost. No, I am not actually trying to answer some question. I am gathering data. If I test the reaction time of X persons -- all in the same way without any variables -- then I am doing an observational study. I don't see why I would need a hypothesis test. After doing this study, I would have data on RT. But if I only tested one person, the study would intuitively not be accepted as valid by most. But if I tested 10K people, it would be accepted as valid by most. In statistics, what is the actual number of test subjects that would make this statistically valid? – A J Jul 19 '15 at 04:46
  • @AJ part of a valid study is having a well formulated objective. Simply gathering some data is not a study. The fact that it is observational is not important.....epidemiologists deal almost exclusively with observational studies yet they always have some objective. –  Jul 19 '15 at 10:22
  • @AJ so..:absent a well formulated hypothesis, your study or data collection exercise will be statistically "valid" if your measurements do not distort whatever you are measuring. How many observations would be desired? As many as possible! There is no theoretical basis for a sample size, but 30-50 observations would support some simple analyses if the data are well behaved.... –  Jul 19 '15 at 10:26
0

These wikipedia pages should be useful to you: https://en.wikipedia.org/wiki/Statistical_significance and https://en.wikipedia.org/wiki/P-value. Also Kahn Academy is always good too https://www.khanacademy.org/math/probability/statistics-inferential/hypothesis-testing/v/hypothesis-testing-and-p-values (this is a great example).

Basically you (or your boss/teacher/someone) determines what "statistical significance means. Often "p-values" of 5% or 1% are chosen (95% or 99% accuracy); though in other contexts they are chosen as $n\sigma$ (often $n=3$) where $\sigma$ is the standard deviation.

Mark
  • 1,339