Saturday, March 03, 2012

A note about SRM 537 and 540 money prizes

I have change blindness. I've seen the rules about distributing money in SRMs so many times that I didn't notice that the rules for SRM 537 and SRM 540 are drastically changed from what became usual in topcoder money matches.

  • Room assignments within each division will performed randomly, with prizes distributed evenly among all rooms in each division.
    56% of the total purse will be awarded to division one competitors, 24% to division two competitors. The remaining 20% of the prize purse will be awarded to 20 competitors chosen uniformly, at random, from both divisions.

  • Approximately 20 competitors will be assigned to each room.
    The first place coder in each room will receive 100% the room award.

  • In the event of a tie for any prize winning position, the sum of the awards of the tied competitors will be distributed evenly.

  • Prizes will only be awarded to competitors who finish with greater than zero scores.

We can assume SRM 537 and SRM 540 to be similar in number of registrants for each divisor to SRM 533, which was recent, also on Saturday and got full registrants. In that match, there 36 division 1 rooms and 52 division 2 rooms. Under the new rules, that would translate to 77 USD given to division 1 room winners and 23 USD for division 2 room winners. Note also that the random part of the prize will be 50 USD.

Something that I do not like about the method is that there are usually a lot more division 2 coders with positive scores than division 1 coders with positive score. This means that if you are in division 2, you will be a lot more likely to win the random prize.

The good thing is that at least this time room winners would get more than the editorial writer. Which would not have been true for the previous rules. As a comparison, I estimated today that with the old rules, first, second and third room places in division 1 would get 43, 26 and 17 USD, whilst first and second places ind division 2 would get 18 and 12 USD.

Giving money to room winners only is not the fairest of strategies possible. I mean, what if you are put in the same room as Petr? You are not going to get any room prize :). The random 50 USD will kind of compensate that.

With some (understatement) luck, I might get first in room and the 50 USD prize, that would add up to 127 USD, which is not bad at all. Will really look forward to this match.

In my opinion, the optimal way to do this would be to give everyone loterry tickets for 100 draws of 50 USD each (Or something like that). And give coders with higher rank more lottery tickets (IE: All coders with 0.0 score receive 1 ticket, all coders with negative score no ticket, first place 100 tickets, etc.)

An approach that is almost completely random does not sound very fair at first. Note however, that all the approaches used are random. The room assignments can really change the outcome of the match. Not only because a room could appear such that only blue coders are active in it. But also because of the challenge phase. There have been matches in which some people got 10+ succesful challenges whilst some rooms didn't have any wrong solution. During a match, there is a lot of random noise, and I think that trying to give prizes deterministically increases the effect of the random noise. It is better to control the random noise instead. But this is all just an opinion. There is also the fact that money prizes are there probably more to attract participation rather than reward top places, a totally random algorithm would do that just as well. If the random algorithm gives more tickets for better ranked players, then with time the better coders would receive more accumulated money than the worse coders. But it is not very practical until we return to the day in which money SRMs were the rule rather than the exception...


Egor Kulikov said...

I really think more fair distribution would be just to have prizes for the top 10 in Div1 and that's all. Nobody seems to complain about marathom match prizes, why all that random prizes and prizes per room, I can't understand

vexorian said...

"Nobody seems to complain about marathon match prize"

Heh, what is funny is that I actually did in the past. Surely it is the most fair approach (albeit there is always a random factor). But the point of prizes is not only to be fair but to attract participants. In that regards, when Marathon announcements come touting that there is 5000 USD in prizes, it sounds a tad ridiculous once you understand that it will take a herculean effort to get 250 (And I can't think of a (worst work/money received) ratio ).

Perhaps go for hybrid approaches, but the problem with that is that it means less prize pool for both ways to earn.

Maybe In this match it would have been better to give the same prize to top 30 instead of giving it to room winners.

Which reminds me that one of the reasons SRM prizes are based on rooms is the challenge phase. People that end in a room with 12 wrong 250 solutions are more likely to get a higher score than those who end in a room with only 1 wrong 250 solution (And this is not a rare occurrence at all).

Egor Kulikov said...

Well, I can think of no other competitive sport where prizes would be distributed in such strange manner
As for randomness of room composition - well, there is random factor in each sport, for example, if you are sprinting for 100m and runner on a neighbor lane stumbled and you then stumble upon him there would be no rerun for you

vexorian said...

There's a random factor in every sport, but the extent of the factor in TopCoder is arguably stronger than the probability a neighbor will stumble upon you during a sprint.

I don't like calling TC a sport. But in most of the real sports competitors are paid for competing as well as for winning so the comparison does not work that well. And there are enough motivators to participate. TC is instead a private company and they are more interested in attracting competitors than in awarding performance. That's the reason behind giving prizes to division 2 at all.

I would now prefer a hybrid approach, like awarding top 20 with something and then distributing a couple of random prizes giving more lottery tickets to the higher positions. Some sort of random prize is needed to attract participants.