0

I have been trying to come up with a formula that would allow ranking competitors according to their success at solving certain tasks, but with a number of criteria in mind. Maybe something exists that I don't know of. Here is the description of what I'm trying to do. Think of it as a competition where each participant can create new tasks for others to solve.

  • Tasks have different difficulties and therefore earn solvers different number of points
  • Participants can create new tasks of varying difficulties if they want to.
  • Participants aren't allowed to solve the tasks created by themselves but should not be penalised in the final ranking.
  • Contributing by creating a task should not have any positive or negative impact on their rank.
  • It is necessary to fairly calculate ranking for someone who creates a very hard task vs someone that creates an easy task.

What I was thinking of doing is get rid of the concept of points and replace it by "percentage of completion". But then how to take into account tasks difficulties? I thought about thinking of difficulty as a % multiplier and normalise according to the number of tasks the participant is allowed to solved. For each participant, getting 100% completion would mean solving all tasks except their own. But I guess it creates a big flaw: what if a participant creates several hard tasks that nobody manages to solve? Could they win the competition by solving a few easy tasks and creating a lot of hard ones?

I'm really struggling here and not sure that it's even doable. Help would be much appreciated!

jrm
  • 101
  • Are the difficulties of the questions known or have to be computed too? And just to confirm, you want people's scores to be higher if they solve harder questions; but creating a task (hard or not) never gives any points? – user3658307 Mar 06 '17 at 15:28
  • The difficulties of the questions are known. They are set by the questions creators and approved by a jury. Solving a question gives a number of points according to the difficulty. Creating a task does not give points. It's completely optional. – jrm Mar 07 '17 at 07:59
  • Ok. I'm not really sure what is wrong with ranking by a point system, where points are weighted by difficulty then. Are you trying to incentivize hard or easy question creation? If so you could consider giving bonus points (depending on the difficulty) to those who create questions if, say, others can solve it (kind of like StackExchange). Also, what does this have to do with the uniform distribution? – user3658307 Mar 07 '17 at 15:36
  • You're completion version sounds ok too: it should incentivize people to create hard questions right? So participants with higher completions are unlikely to have just "solved a few easy questions" because (1) no one will want to create them and (2) the denominator (number of tasks they are allowed to solve) should be fairly high (since everyone will be incentivized to make hard questions, to reduce others' completion rates). Perhaps I am misunderstanding the setup though. – user3658307 Mar 07 '17 at 15:38

0 Answers0