In cook-off Jan2021, server issues were on peak , not even submit button was working, and it was rated .
In lunchtime 2021, server issues were there but then also paticipants were able to submit the solutions. If we see solutions of A,B,C,D,E and F , they are the usual number of submissions as a lunchtime has. Participants who have performed well and invested time what about that ?
Rated and Unrated should be based on server issues and submissions . Submissions are going on in these server issues so no point in making it unrated .
If Jan cook off was rated then this should also be rated .
If it was to be made unrated then it should have been announced during the contest .
Cheating has also increased in long contest , only short is then left to measure your ability , and now short is made unrated and long is rated . Then i have one question what does rating of codechef shows ?
This should be made rated else people will loose faith on giving time on codechef and feels very demotivated .
I have discussed the rationale behind making LTIME unrated here.
The contest was rated until 11:15 pm, but ever since then, the queue kept piling on, and by the time contest ended - solutions submitted at 11:30 did not even have a verdict. How would it be fair for it to be rated?
I’m sorry, I know it sucks and I don’t want any contest to be unrated myself, but in the end - we just faced too many issues at once today, and making it rated would be unfair for all the contestants who suffered.
As you must have read, CodeChef is taking a break from short rated contests until the issues are resolved, so hopefully this is the last time this happens!
TL;DR: Queue issues were the breaking point for the contest - not the server issues.
Was the queue actually a queue?
I noticed that few submissions were randomly not getting a verdict while most others got a verdict.
Even when I myself did not get a verdict for 10 minutes, I made another submission and got a verdict within 1 minute. (all this is after 11:30 btw)
So, if the queue was the same for everyone, can we say the contest was fair?
If for some reason the queue was different for different people, (pardon my limited knowledge of queues but usually I think of FIFO) then I agree we have a problem, but rather than rolling back to cancelling the contest, we might as well consider rolling back to T=11:15 PM since its agreed by the admins that it was the last point where the contest remained fair?
This was something weird. One of my old submissions was in queue for around 10-12 minutes and my latest submission had already got the verdict. When I clicked my old submission, I could see the verdict but it was shown (in queue/running) in my profile.
Yes, it should be rated.
This was my first contest on CodeChef and that too announced unrated after the contest.
This is not fair at all, because the problem raised for every participate during the contest.
Please make it rated.
During the first 30 mins, it is giving the same not found an error again and again but after 30 mins everything is fine. So it is hard to decide whether it should be rated or not because in any case, it’s is unfair to some participants.
At the same time, it is very shameful that CodeChef can not even handle around 6000-7000(approximately) people during a short contest.
So, I think codechef must be using a message queue for their system and you are right about the FIFO but there is not a single instance running on the cloud there are multiple servers(horizontal scaling) now there are load balancers in place for evenly distributing the load so that one server is not significantly ahead of the others(consistent hashing is an algorithm which does that) but as codechef said they are having architecture issues that’s probably the reason why people submitting solutions later getting verdicts faster because the server where there answer went was working while the other server had a long queue or was just down because of the load or any other reason, deducing anything more than this would be speculating for us without knowing how codechefs system works.
Good point.
It seems to be more of a problem in the implementation of the queue compared to anything else.
Even at 11:50 P.M, most submissions got a verdict in 1 minute or so. There are few submissions which probably went into queues for a long time while others were judged instantly, this implies that the queue manager/ load balancer was not working correctly (which is even funnier and a bigger architectural problem than just all queues having wait).
If a solution A is in a queue for 10 minutes and other solutions can be judged in one minute, this implies that there is at least one queue with 1 minute wait time, and the solution A should be moved there instead first.
So i suppose its not technically a mere queue load issue(as has happened in the past where some test case was very slow) but more of an architecture/ load balancing/ resilience issue. If we assume the architecture has not changed over the past few months to facilitate this issue, this issue must also have arisen in past contests.
I am still not convinced why single this contest out, unless there is some recent architecture change that caused issues that arose here. If so, i will understand.
. Since lunchtime accounts for when the score was achieved this is a valid strategy