2

Background: I have a website where students vote on the attractiveness of their peers: they are presented with two images, and they must pick one (the "winner")- then the elo score for each is updated. A user can choose to vote globally (i.e. cross population, i.e. between all schools), or a user can vote within a single school (i.e. separate population). All students start with 1200 points, there is a minimum of 1000 points. It's easy to rank students individually: for example the most attractive person within school "x" is simply the one with the highest elo points.

Problem: I want to have a ranking for a group of students- in other words i want to say "here are the most attractive schools"

My idea (which I think is bad): Take an average of the elo points for all students. My thought is that since "every win also means there is a loss" the distribution curve for all schools should be relatively similar- so the average elo points for every school will be relatively similar. A wider distribution curve would simply mean there are greater numbers of both more attractive and less attractive students (I think?).

So does anyone have a (somewhat statistically significant) method of saying "this group of students is more attractive than that group of students?" I feel like I'm not collecting the proper metric- this would be easy if the users were ranking a single student on a 1-5 star scale, rather than my system where you're asked to pick between two students.

Any advice much appreciated, thank you.

jaredrada
  • 21
  • What's the problem of taking average?

    1. If the user votes within the school, no average values are changed.

    2. If the user votes between schools, then an average of one school becomes higher and of other - smaller.

    So, if average elo points for schools are similar then they are similarly beautiful.

     – felagund Mar 06 '14 at 08:36
  • I expect the vast majority of votes to be within a school (>99%). So the average of one school should be similar to any other, correct? – jaredrada Mar 07 '14 at 01:33
  • Yes, in this case less than 1% of the votes will change the average of any school. – felagund Mar 07 '14 at 13:19

1 Answers1

0

I can't think of reliable way to compare schools in the case when 99% of votes are within one school. May be elo score is not appropriate in this case. I can suggest the following way to compare two schools:

1) If the averages of schools differ significantly, then use them.

2) If no, then use the following statistic: the percentage of students who have rating more than some threshold (by percentage I mean normalizing on the number of students in the school).

felagund
  • 92