Cross-posted from Elo-MMR ratings go live on CodeChef! - Codeforces
Huge shoutouts to Aram and Paul for patiently answering all our queries, and helping us in the transition over the last 4 months!
5 Likes
Less than 800 additional users have CC - MM > 200, and none of these are over 500. Nonetheless, these users’ ratings will most likely decrease over their next few contests.
I assume these are mostly 6&7 star users. I fail to see why you don’t adjust their ratings now. If they don’t participate, they will end up having a better rating than those participating. Which means those that participate in contests are indirectly punished.
I would have preferred if you replaced the CC ratings of all users with their Elo-MMR. It may be confusing in the beginning, but your approach seems to be confusing long-term. Which is worse.
6 Likes
Precisely, it does not make sense to me. People can just choose to not participate and have good ratings which sucks
7 Likes
Among those ~800 users, less than 25 are currently 6 or 7 stars.
Everyone’s rating will be force-changed to MM after a while.
2 Likes
Seems like this ELO system draws a lot from past contest history . renatyss’s MMR jumped from 2339 to 2434 while my MMR changed by 2271 to 2350. Even though i had a better rank than renatyss in today’s lunchtime. So you can get a greater delta from a higher MMR even with a worse rank. Why are these two rating diverging apart from each other in opposite direction?
4 Likes
hii admin i have found new problem in codechef , due to new interface i have forgot that is discussion forum for every question i mean in this present new interface it trown corner names “discuss forum” in solution tab where as in old interface it shows there at solution section it is really visible . please continue discussion forum below editorial instead for new link . not only me there are so many people found this difficult to remenber ther is discussion forum . you can observer there is less discussion happening with this new interface when compared to old . discussion forum really very helped to learn new ideas and techniques where i have gone in my code now i am seeing no discussion please rectify this issue
2 Likes
what if someone’s MM > CC , and he choose not to participate in any contest . will his CC going to be replaced by his MM also.
1 Like
Yes, the internal state is a bit more complicated than just \mu,\sigma. It’s likely that your friend had already acquired evidence of being at the higher skill level, maybe due to some recent high performances. The system assigns less weight to outlier performances, but may increase that weight if repeat wins demonstrate it not to be a fluke.
4 Likes
Yes - “After a period of time (less than a year), ever user’s (including those who aren’t participating in contests now) rating will be changed fully to Elo-MMR rating.”
1 Like
Faster convergence for experts: Compared to the previous rating system, Elo-MMR awards a much higher increase to users who do well in their first CodeChef rounds.
I can’t call myself an expert, but I was not a complete beginner when I participated in my first CodeChef contest. Still my performance was far from optimal (rank 302 in division 3), largely because I wasn’t familiar with the UI of the CodeChef platform during that contest. Now I wonder, how much did this blunder affect my curent Elo-MMR score? Does Elo-MMR quickly adapt or does it assign a high weight to the first contests, which sticks longer?
Also even complete beginners may improve really fast after only a few contests thanks to learning new tricks and algorithms (there are a lot of low hanging fruits). Does the new rating system now penalize beginners, making the results of their first contests more important?
1 Like
The weight of the first contests is not increased. Instead, the weight of the provisional starting rating is decreased. Glicko and TrueSkill do essentially the same thing.
1 Like
The weight of the first contests is not increased. Instead, the weight of the provisional starting rating is decreased.
Isn’t this basically the same thing? What you are saying sounds like the new system essentially replaces the initial provisional rating with the performance of the first contests. Making the first contests very important, considering that the system is also “Robust: players will never lose too much rating for a single bad day” (“players will never gain too much rating for a single good day”), which may indicate that the system adapts slowly after the initial bootstrap.
If the system converges very fast for new accounts but then adapts very slowly, then this may provide a strong incentive for accounts rerolling (to get a more favorable starting position).
Could you please calculate Elo-MMR scores of CodeChef users after discarding the results of their first contest and share the results on drive? I would like to confirm whether the impact of the first contest still persists many contests later or not.
BTW, looks like your account may be another example of an expert (you were candidate master on codeforces in 2012), who happened to have bad performance in their first CodeChef contest.
3 Likes
It’s not the same thing: a user who has done a lot of contests would have almost zero weight on either the provisional rating or the first few contests. For such a user, feature #8 is completely irrelevant.
If you’re worried about robustness, the system does have a bit of a memory. A player who is consistently overperforming (or underperforming), or who performs inconsistently, would be allowed larger updates, whereas a very consistent player’s rating would be less sensitive. It’s basically a gentler version of the “volatility” parameter in the older systems, but carefully designed to retain incentive-volatility.
2 Likes
Did you consider that the system may grant too high of a “newbie” bonus?
look at this user:
#24 Div4 and #5 in Div4. 2 contests total and his MMR is 2166. I can easily score better than him but I cannot reach 2166 MMR. Reaching almost 6* with only Div 4 contests can’t be correct.
8 Likes
To be fair, this user wouldn’t normally have been placed into Div 4: their Elo-MMR rating after the first contest was already too high.
The #4 placer is a better example of a user who got a huge boost on their first contest, to 1966. Is that too high? Well, keep in mind that Topcoder rated Petr at 1866 after his first round, which was half as long as a CodeChef round (and I think they use a smaller \mu of 1200 instead of 1500, though Elo-MMR has the 3\sigma penalty from feature #7 instead…)
Now, I don’t have enough experience with Div 4 contests to judge how fair this is. The rating systems are not aware of problem difficulty; they just assume that the problems are sufficiently calibrated to accurately rank all the contestants. If the user had done well in Div 2 or 1 instead, I imagine you’d be less surprised.
One approach we could take is to cap performance scores from Div 4 contests to something like 2000. I don’t know if that’s necessary, but it would be interesting to see if these users actually turn out to be overrated when they later go on to compete in Div 1-2.
6 Likes
@admin want to say something??.. Now everyone will make a new account and get a 5 or 6 star rating with ease. All that matters is the heighest rating, which we will write in our CV. Soon CC rating won’t be considered for evaluating someone’s skill.
10 Likes
Watch his profile. igor_y.
He almost went to 5* just for getting Rank 1 in a single Div.3 Long Challenge.
Is it really good to have so much rating change in a single contest?
Shouldn’t there be an upper limit otherwise everyone would just make a new account and try to TOP in Long Challenge ?
5 Likes