2

I want to compare means from samples of varying size, plug them into a bar graph and have appropriate values to plug in for the error bars. My sample sizes are:

A = 1
B = 3
C = 14
D = 7
E = 13
F = 190
G = 33

I intended to use the typical standard error calculation 

Standard Error = Standard Deviation/number of sample^1/2

but because many of my samples are so small, a colleague suggested I use bootstrapping to compute confidence intervals instead. He's smarter than me so I found some software that did that (scikits.bootstrap==0.3.1 for python in a pandas environment), computed my bootstrapping confidence intervals and used them to put error bars on my graph.

What I want to know is WHY exactly bootstrapping is more appropriate than standard error in this case? I understand that bootstrapping works by creating many alternative versions of your sample by sampling-with-replacement and computing some descriptive statistics on that large population of hypothetical samples but I don't understand when and why you should use it instead of using the standard error.

NOTE: I'm not applying these calculations to sample A because its just a single value; I'm actually dismissing it from consideration. I only included it here because it was easier to copy and paste the whole list rather than remove A and change all the letters

Slavatron
  • 145
  • How are you calculating the standard deviation? With a sample size of 1 (like in A) you will not have a standard dev. Is it just an assumed value? –  Jul 17 '14 at 02:30
  • Yeah, I should have clarified: I'm not doing anything with that sample because its only got one measurement. – Slavatron Jul 17 '14 at 05:37
  • Please tell me why my question has been put on hold so I can make appropriate edits and avoid making the same kind of mistake in the future – Slavatron Jul 17 '14 at 14:39

1 Answers1

2

With your small samples, your estimate of standard deviation will be very biased (low). With bootstrapping, you can approximate the sampling distribution of your estimators by treating your sample as an approximation of the true distribution of your data. The more samples you have, the closer your empirical distribution will be to the actual distribution, hence allowing ever more accurate error estimates.

That is why boostrapping works. The standard deviation approach only works when you know the true standard deviation and the distribution of the estimator is approximately normal. Boostrapping is just monte carlo simulation using your data as the input distribution, and so the more representative your sample, the better your inferences.

Also, I would highly recommend a parametric boostrap for samples <20 points. This requires assuming some underlying distribution for your data and then simulating boostrap samples from the fitted distribution. If your data are strictly positive, then try a lognormal, if they can be both positive and negative, try the normal distribution.