Elo-MMR ratings go live on CodeChef!

To be fair, this user wouldn’t normally have been placed into Div 4: their Elo-MMR rating after the first contest was already too high.

The #4 placer is a better example of a user who got a huge boost on their first contest, to 1966. Is that too high? Well, keep in mind that Topcoder rated Petr at 1866 after his first round, which was half as long as a CodeChef round (and I think they use a smaller \mu of 1200 instead of 1500, though Elo-MMR has the 3\sigma penalty from feature #7 instead…)

Now, I don’t have enough experience with Div 4 contests to judge how fair this is. The rating systems are not aware of problem difficulty; they just assume that the problems are sufficiently calibrated to accurately rank all the contestants. If the user had done well in Div 2 or 1 instead, I imagine you’d be less surprised.

One approach we could take is to cap performance scores from Div 4 contests to something like 2000. I don’t know if that’s necessary, but it would be interesting to see if these users actually turn out to be overrated when they later go on to compete in Div 1-2.

6 Likes

@admin want to say something??.. Now everyone will make a new account and get a 5 or 6 star rating with ease. All that matters is the heighest rating, which we will write in our CV. Soon CC rating won’t be considered for evaluating someone’s skill.

10 Likes

Watch his profile. igor_y.

He almost went to 5* just for getting Rank 1 in a single Div.3 Long Challenge.

Is it really good to have so much rating change in a single contest?

Shouldn’t there be an upper limit otherwise everyone would just make a new account and try to TOP in Long Challenge ?

5 Likes

@admin when is the drive gonna get updated?

5 Likes

Alt accounts are clearly being created to gain a higher rating…old accounts don’t gain much for the same rank…
Newbie gains are not necessarily newbie gains.A rating system that has such a loophole makes it a deterrent for people who don’t resort to such tactics. I think “fairness/robustness to cheating” is the most desired variable.

8 Likes

Update:

We have introduced Provisional Ratings, which signify that a new user’s ratings aren’t very indicative of their actual rating for the first few contests that they participate in.

In particular, until a user participates in at least 5 rated contests, their rating will be accompanied by a question mark, indicating that it is a Provisional Rating.

95 Likes

Thanks, that’s the move in the right direction. Could you please also ensure that the users with provisional ratings don’t get star badges assigned to them in this forum and in other places until they finish their 5th rated contest?

26 Likes

nice suggestion. :+1:

6 Likes

I am new to CC. Can I know why this was necessary?
I mean, how CC was getting affected before this update?

4 Likes

By this what changes should be done in Provisional
Rating Means In Provisional
Rating rating they get is high (if they solve more problems) +ve delta as compare to Normal Rating

3 Likes

I am new on CodeChef. I don’t know what is meant by provisional rating.

7 Likes

Just participate in 5 contests, and the question mark next to your rating will be removed.

27 Likes

I’m quite confused and disappointed with what’s been happening with the rating in Codechef. Yesterday after the STARTER50 my new rating on my profile was changed to 1913 from 1882. Today it changed to 1893 without participating in any contest. Even though ranked 11 globally my overall rating only changed 11 on profile! But I just checked the file ‘changes_after_starter50’ from the drive link. It shows the current elo_rating is 1802 and the previous elo_rating is 1836. I don’t get the math here. This whole transition is a mess.

14 Likes

Update:

After launching Elo-MMR, we got feedback from a lot of users about the sudden increase in ratings of new users, particularly in their first couple of contests. This was supposed to be a feature, but it had been a bit too much. So now, we have introduced a change, in which the weight of the first and second contests of a new user is changed to 60% and 80% of their old values, respectively. This leads to a slower increase in ratings of new users.

Along with Provisional Ratings, we hope this leads to a more robust rating eco-system.

This change has been applied for all users and their Elo-MMR rating recalculated from the beginning of time, and hence most users will see a change in their display ratings.

A comparison between the previous and current ratings (display ratings):

  • 31 users have an increase of more than 50
  • ~11700 users have an increase of between 0 and 50
  • ~465000 users have no change
  • 160 users have a decrease of more than 100 (70 of whom have been caught in cheating)
  • ~4000 users have a decrease of between 50 and 100
  • ~32000 users have a decrease of between 0 and 50.

You can find the entire list of users with their rating changes here.

6 Likes

can anybody explain why I got only 27+ rating in July Lunchtime 2022 Division 2 (Rated)
with 191 global rank
previous rating 1628
@admin

4 Likes

@gouravcc1 - you can check out this thread.

1 Like

it happened with me to with 191 globle rank in july lunchtime I got only 27+

1 Like

I would say & support that codehef’s old rating system was good rather than current system . Because i think after giving a good performance than previous contest doesn’t brings any such change , even worst performance doesn’t brings any good decrement .
For 4* , 5* coders to get a gooding rating is now huge difficult than codeforces as well .

7 Likes

My codechef graph has changed…
Some time before I had participated in several competitions and most of my graph points were in 1400-1600(green) range but now the graph is showing those points to be above 1600(blue), Is this normal?
Can anyone explain why did this happen?

3 Likes

@demonslah - so 2 things done in the last month

  • plagiarizers were removed and ratings recalculated for all users
  • ELO MMR introduced

This thread should give you an idea of th echanges.

1 Like