Unfair tagging of codechef problems

So after watching this for a while now, I realise somebody needs to mention this so that the admin notices. Most of the codechef problems from the official contests are tagged in an extremely unfair manner where much harder problems are given Easy tag.

For instance, see this - https://www.codechef.com/COOK117A/problems/KPERM, a problem from the last CookOff, and I am sure most will agree this is not a trivial problem. Also, the same is evident from the number of submissions during the contest.

This is not the first one, I am sure there are others as I have been watching this for some time. Apparently, the problems are categorised (easy/medium/hard) based on the concept it revolves around. So, yeah sure permutations are considered something that beginners can touch but that’s not it because not all problems around permutations are easily solvable unless you are able to find that hidden insight/observation to it.

Another one is DDQUERY. Again, I don’t know how a problem which needs at least medium difficult tag is classified under Easy section.

I strongly encourage the admin to rethink whether a learning platform should allow such incorrect classifications and whether just the concept used to solve the problem is enough to decide how hard it is.

6 Likes

The problem difficulty tags are given by tester and verified by contest admin/setter/editorialist. Some problems can be on harder side of difficulty, but I need more instances to initiate a discussion.

Any other issue aside from difficulty tag?

I will look at Medium tagged problem ending up in easy section.

2 Likes

It’s just about the difficult tag.

I think the difficulty tags should not be given by setters at all and there’s a strong reason for this. The setters are sometimes not aware of how difficult the problem is because the concept that the problem revolves around was already present in their mind when they designed it, which might not be an easy-to-strike concept for rest of folks who are reading it for the first time. As it has happened quite a few times on Codeforces as well, the problem B was more difficult than C (div.2) because of a hidden observation which was obvious for testers/setters but not for the contestants and the total submissions on B were much less.

To find more instances of this issue, please visit easy section and see problems with lowest submissions.

I see the difficulty tagging as a hard problem itself. If we use a priori tagging (before the contest), it would be necesary based on the opinion of few people. Labeling based on the number of accepted solutions or precision has its own drawbacks: for example, it depends on how long the problem has been available, or the length of the contest or if there have been problems with similar approaches close to that. So … I don’t see an easy and reliable tagging method. What is your suggestion?

4 Likes

They tag difficulty according to them, not acc. to contestants. :stuck_out_tongue:

The difficulty tag is not given by setter, but primarily tester. That helps in avoiding the setter bias.

The thing is, number of submissions cannot be a true estimate of difficulty because often an easier problem, if a little implementation heavy or involving some trick/observation, is mistaken for medium or hard problem by contestants and skipped. Same if the problem involves implementation of something non trivial - despite being easy it’d not be widely attempted.

Regarding why that problem was moved into Easy section - the problem was actually of easy-medium difficulty. It took that difficulty slot for the problem. After submissions/contest, perhaps the contest admin or editorialist might have decided that the problem is on harder side of easy-med (and hence tagged medium difficulty in editorial), but since the problems are to be moved to practice section immediately, it went into easy section where all the easy-med problems go.

While I see your views on how this process is indeed flawed, I feel these errors are rare. If their frequency is low, its not really worth the effort for us to right now brainstorm some better strategy (but in case you have suggestions, do let us know). Hence, we will not be taking any action unless the frequency of such cases increase.

2 Likes

Right, it depends upon how long the problem is available. But then for a particular contest it would never be a case that the said problem was added halfway into the contest and not in the start. At least, it won’t happen in the official contests. So just consider the submissions received during the conetst.

As for the length of the contest, other platforms do have only short contests like Codeforces and I can see most of the problems have fair tags. Even if the contest is short, why would it matter? You see, the number of submissions in most of the cases is proportional to problem difficulty. I am not saying that a random person is allowed to choose the difficulty tags, but let them be decided based on cumulative suggestions of tester+setter (or include even the editorialist). But let them make a change in their decision post contest. So let’s say if they gave Easy tag to some problem which actually received much less than expected submissions, they can move it to Medium section after discussion with each other.

Thanks for considering there could be errors in the process of assigning tags.

Just consider the option that post contest, editorialist as well as tester and setter can discuss among themselves and change the tag of the problems based on number of submissions received. At least, give a chance to make this change when contest ends.

https://codeforces.com/blog/entry/76635

Just ask mike to open source his earlier formula. In general we dont have much div2 only contests. So earlier formula should be enough for all cc needs.

2 Likes

I will forward the feedback to admin. Beyond that, its really on discretion of contest admins but I trust they will do their best :slight_smile: