2

We need to track student advancement in a topic based on pre and post test scores. That is, we give a pre-test on day 1 of class, then on the last day we give the exact same test, renamed as a post-test. Somehow we have to take the first score and assign a target number for the second number to show improvement. This second number has to be higher than the first number. Students who hit this number, or above, on the post test are considered a success. Students who do not are counted as a failure. Based on the percent of students who are "success" we get evaluated as how well the students learned. My problem is trying to figure out how to calculate how well the students did, or did not, improve.

My first thought was simply take the difference in scores and calculate a percentage change. But how do I then measure student success? A change in 50% of their score? This seems somewhat easy for people who did rather poor on their pre-test, but those who did rather well it seems to penalize. That is, jumping from 0% to 50% is great, but jumping from 75% to 85% is does not seem as good if we just measure %change. The first student would be a "success" even failing the course, but the second student would be considered a "failure" even passing the course rather well.

If it helps, here are the pre-test scores for one class:
20 40 44 12 48 32 24 44 28 0 36 40

and pre-test scores from another... 76 40 40 32 60 64 68 48 36 72 56 20 24 36 52

the exact same method must be used in each class to show "success" versus "failure".

Michael
  • 63

1 Answers1

1

With educational research, unless you have a very well-calibrated test, the quantitative differences (i.e., magnitudes), are not overly informative. However, a useful measure is the fraction of students that "didn't fail". Define your "success variable" ($S$) as follows:

  1. For each student $i$, let their pre-test score be $I_i$, and postest score be $P_i$.
  2. Remove any students that had a perfect pre-score - they will not give an accurate value.
  3. Assign each student a success variable $S_i$ which equals $1$ if $P_i> I_i$ adnd $0$ otherwise.
  4. The success rate for a class of $N$ students is simply $\frac{\sum S_i}{N}$

This method avoids the "scaling" issues when, for example, you have a very high performing class, so there is little room for improvement. In addition, it doesn't over-emphasize the amount of improvement, which is more doubtful (unless you have a very well calibrated test).

Anyway, this may help you make some progress.