"Judge Error" Issues in Interactive Problems

As evident from a lot of “Judge Error” verdicts in today’s problem Secret Tree, this problem from December Lunchtime, this problem from Snackdown Finals and almost any interactive problem in recent contests , there is something seriously wrong with the way the interactive problems are judged on this platform. Most participants think that it is due to bad judge implementation but that’s not the case. The problem setters, testers, contest admins, editorialists all put in a lot of effort during the testing phase to make sure that there is no mistake in such things and it not possible that almost all interactive will have some bugs causing the “Judge Error”.

According to my observations, one of the most common situation in which this verdict is given is when the solution must get the “Idleness Limit Exceeded” (as per Codeforces) but the Codechef system is unable to detect an ILE and thinks that the judge got TLE (by obviously waiting infinitely for a query by the participant’s solution) and declares it to be a “Judge Error”.

This feature bug can be quite annoying for the participants because if they would have got the ILE verdict, it would have been much easier for them to debug compared to the random “Judge Error”. On the other hand, no penalty is added for this verdict whereas an ILE incurs a penalty on Codeforces. So, this may also turn out to be advantageous for them depending on how quickly they manage to detect the bug. Whatever may be the case, it is a serious and frustrating issue that needs to be resolved as soon as possible.

2 Likes