I am not sure if this would be considered a programming or a math or another problem. I've come across a situation in my day to day work and am looking for an optimal solution.
If you have a list of rankings from experts (let's say players in a gaming league). Not every list has the same players (but there's some common overlap) and not every list is the same length. What would be an appropriate technique to join the lists to a single one that reflect the generate group consensus?
Small example:
List 1: 1. A 2. B 3. C
List 2: 1. A, 2. C, 3, B, 4. D
The final list may look something like: 1. A. 2. B. 3. C. 4. D
A few solutions I've looked at are:
- Elo or Glicko - using the matches (from the lists above as as an example): A beats B, A beats C, B beats C, A beats C, A beats B, A beats D, C beats B, C beats D, B beats D.
- Swarm Analysis / Particle Swarm Optimization - find a weighting that minimizes discrepancies between master list and each expert's opinion.
- Simple weighted average - problems run with not all lists are same length nor do all lists include all players
- Ranked Average Voting (such as in STV voting systems)..
What are others, or what would be the best technique for this problem? By best I mean what produces a final list that best reflects group consensus?
