Which statistical test to choose

Question

first I will describe my problem (1), then solutions I found (2) and then pose the question (3).

1) I am a student making simple program for statistical analysis of data for biological laboratory. The situation is as follows: They have a protein and measure some property three times (to eliminate measurement error). Then they mix the same batch of protein with some chemical and measure the same property again twice. In the end they want to know, if the measurements are different enough. (I know the number of measurements is low, but it is the maximum they can do).

2) If we assume, that the samples are independent I would choose Welch's t-test, because we cannot assume the same variance. However they say that, due to the fact that we measure the same batch of protein three times, and then two times with some chemical, the before and after measurements are related samples. But for related samples I only found paired t-tests, which cannot be used here, because there are no explicit pairs. The only thing that comes to my mind is to pair each measurement from each set.

3) How to measure statistically significant difference of a property, when we have 3 measurements of batch of protein before applying a chemical and 2 measurements after applying a chemical. Measurements are repeated to eliminate error in measurement device.

Answer 1

This semi-answer used to be a Comment, but too abbreviated to make sense:

Everything "they say" is not necessarily helpful. I think a two-sample t test is the right thing to use. The two groups in any two-sample test have something in common. Details are sparse, but the argument that you don't have two independent samples (as presented) seems pathetic.

However, Welch 2-sample t will probably not find a difference with three and two replications per group. Even with a pooled 2-sample t test, you're down to three degrees of freedom, so the absolute value of the t statistic would have to be large to get a significant result. A Welch t test will characteristically have even fewer degrees of freedom.

If all of the before measurements are smaller than any of the after measurements (or the reverse), then you'd have a P-value of only $1/{5 \choose 3} = 1/10,$ based on relative sizes of the measurements. So no kind of rank-based test (or permutation test) is going to find a significant result at the 5% level.

Recommendation: Cross fingers; use 2-sample pooled t test; hope for the best. The two samples are based on testing the same medium, with the same method and with the same chemistry students students using the technology. Maybe OK to hope for equal population variances.

---

Here are the Welch and the Pooled two-sample t tests done in R statistical software with some "data" I made up--with the idea of maybe being able to show a significant difference. Welch not significant at the 5% level; Pooled is significant.

x = c(12.173, 13.531, 11.044);  y = c(17.681, 21.321)  # My fake data
t.test(x,y)  # Default in R is Welch

        Welch Two Sample t-test

data:  x and y 
t = -3.7058, df = 1.32, p-value = 0.1188  # P-val > .05: Not Signif
alternative hypothesis: true difference in means is not equal to 0 
95 percent confidence interval:
 -21.552561   7.049228           # Includes 0: Not signif
sample estimates:
mean of x mean of y 
 12.24933  19.50100 

t.test(x,y, var.eq=T)  # Parameter 'var.eq=T' forces Pooled test

        Two Sample t-test

data:  x and y 
t = -4.4118, df = 3, p-value = 0.02161   # P-value < 5%: Significant
alternative hypothesis: true difference in means is not equal to 0 
95 percent confidence interval:
 -12.482607  -2.020727     $ Does NOT include 0:  Significant
sample estimates:
mean of x mean of y 
 12.24933  19.50100 

qt(.975, c(1.32, 3))
## 7.311460 3.182446  # 5% critical values for Welch and Pooled 2-sided test ...
                      #  ... |T| > crit. val implies Reject Null Hypothesis

Answer 2

A short hint. Since the number is low and same batch of proteins is used to do two different tasks you can choose t-test $t=\frac {d\bar-\mu }{\frac {s}{\sqrt {n}}} $ where $d\bar$ difference between results of two tests and $\mu,s$ are the mean and standard deviations of the $d\bar $. n is the total number of samples tested. Hope it helps!