Some Problems in the state of Codechef

[randomusername]

TL;DR This site is starting to become really bad and unprofessional for no reason.

-An open letter to Codechef administration, and all Codechef users-

I joined Codechef almost two years ago. Since I’m a student, two years mean a lot, and I have gone a long way since 2011. Codechef really helped. I did my first programming problems here, did 1-2 problems in Codechef Long contests and now I am able to do 6 with no effort. That’s a nice progress, and I’m really grateful.

But what’s bothering me is the Codechef today. I think that things were different back in the days but I might be wrong. I just don’t remember this site to be this unorganized and unprofessional. I don’t mean to be rude, but as one of the biggest Codechef fans, always putting it on the top of my coding websites list and being in love with it for two years, I felt the need to say something about the state of it these days.

The long contests (aka challenges) are the main codechef “attraction”, and yet they are so poorly written nowadays. I’m not talking about bad test cases, or incorrect constrains. That happens everywhere. I’m talking about editorials that are poorly written, and late 3 days. I’m talking about problem statements written in a language that barely resembles English. I’m talking about challenge problem grading that is ridiculous. Codechef is the only place where rating updates last for 5 days along with challenge problem regrading. This site, famous as it is, should be no place for these kinds of mistakes.

For example: Look at the task regarding ciphers in this month’s challenge. What is that?! That statement is ridiculous! Too many ambiguities and errors. That task made me write this, it was the last drop in the ocean. How can you call that mess a problem statement? It took me few hours to get it right, and I’m sure I could’ve spent no more than 15 minutes on that problem if it was nicely written. It’s trivial.

One might say: Well, ok, we need a faster server. I don’t think that’s the problem. I just feel there is no devotion. Your job every month regarding the challenge is to prepare 10 tasks, write them properly, write the solutions, and make test cases. I strongly believe that three people (two programmers and a translator) can do it in a few days, set aside whole month.

Just look at other coding sites and the old Codechef. Challenge problem grading has to be done earlier so the results and ratings can be available few hours after the contest ends. Editorials should also be written on time. I think that I, not a professional programmer, can write a decent problem + solution + test cases in a day or so. People behind Codechef are bigger in numbers, and also professionals. How can they fail to deliver everything properly on time?

Also, the problem balance. It’s a trivial matter compared to other problems, but I think the problems in long contest are really unbalanced.

I’m not insulting, I will repeat again how much I love Codechef. I’m just seeking an explanation because I am very confused. This site is getting more and more unprofessional with each month, with ridiculous problem statements and regrade times. Why is that happening and why is it so hard to try a little more and take care of it all, like 2 years ago?

I am aware of the fact that I will receive indirect and generic responses. I am just trying to snap all Codechef users back to reality and make them realize how bad things are.

Best regards,
RandomUsername

[yashkumar18]

I agree with some of the points, especially some of the editorials are not at all helpful, as a newbie I look forward to editorials after contests but some of them do not help at all, have to search myself all over the internet to learn the new concepts (sometime with no success), not gonna point any finger

[kuruma]

Hello @RandomUsername,

Being on the problem setters’ panel myself, I have to agree with you at least on the part of the problems being unbalanced.

This matter is not new and, in fact, I pointed this out after NOV13 Long Challenge: the way problems are selected for each monthly contest is very “tester dependent”, i.e. , it is very dependent on what problem testers believe to be easy/medium/hard, which of course, can be different from tester to tester and from setter to setter. Obviously, this is not trivial “to fix”, as problem setters’ are even more biased towards the difficulty of the problems than the testers’ themselves. Which is normal, and which makes sense.

If I have never used Segment Trees in my life, and I see it as a somewhat advanced data structure (which it is) , it would be normal that I would see a problem involving it as being a MEDIUM problem, but, on the eyes of the testers’ such “trivial problem with point updates and range queries” is definitely easy. Which is also a legitimate point of view.

However, such disparity leaves us at a “dead end”, and, as such it was chosen to let the testers’ decide the real difficulty level of the problems maybe also considering the authors’ opinion, idk.

The fact is that, this machine is definitely failing somewhere, I think I know where, but, it might not be so easy to fix.

Sometime ago we had no problem classification I think (only like Easy, Medium, Hard, can someone check this?), and, as such, setters and testers didn’t worry too much about it, after all, Cakewalk, Simple and Easy diffculty levels were all grouped into the more convenient label of: Easy.

Ironically, it seems that bringing the more specific problem classification only made matters worse, with some unbalanced contests such as this one and the month before having 4/5 problems with over 1000 solvers and the remaining ones with very few solvers (less that 600 to be exact and with 3 problems having less than 100 people solving it).

Maybe this is happening because setters/testers, team in general, tends to “broaden” the guidelines a lot… As to give a specific example, here are the guidelines:

1 CAKEWALK: The problem should be solved by someone who knows any programming language and basic data structures like arrays and lists. No further knowledge should be considered mandatory to solve these problems. Example: MAXCOUNT.

2 SIMPLE: These may or may not require some algorithms but whatever is required should be immediately obvious from statement, similar solution idea should be easily available in any text book, easily searchable, and more importantly should be very easily implementable. Almost zero genius is required. Example: LUCKY5 and WCOUNT.

2 EASY: This should be fairly easy for most of the contestants to solve this problem without too many optimizations. A novice programmer should be able to solve it within his/her knowledge of programming. May not require knowledge of advanced data structures. Example: RHOUSE and LUCKY1.

2 MEDIUM: These problems should require some more work for anyone to solve them. Advanced programming concepts like DP, Graphs, Trees or Mathematics etc, may be needed to solve them. May also require knowledge of advanced data structures. However it should not involve looking into research papers for the problems to be solved. Not easy to make. A novice programmer should not be able to solve them without putting in lot of effort. Example: BBSYSTEM and CYDB.

2 HARD: These are the hardest to make. They should be able to challenge the best programmers out there for a 10 day contest. The idea to solve should be hard to come up with even with the knowledge of most advanced algorithms. Often, Problem Setters have made use of problems in research to get ideas regarding such problems. Example: FLYDIST, DOMNOCUT, WINDOW, SHORT2 and HYPER.

1 CHALLENGE: All the above problems are binary problems (Either they give a full score of 1 or none 0). They are typically optimization problems (maximize or minimize). The best submission gets a score of 1 while the rest get scores relative to the points of the best submission. The difficulty of getting an accepted solution on the challenge problem should not exceed “Medium”. Ideally it should lie between “easy” to “medium”. Example: THREECLR, ALLINONE, BICKER, CAVE and L2GRAPH.

I guess that considering a problem which uses Segment Trees as Easy, given that it’s an advanced data structure only taught at university level, might be a bit of an overkill and I really don’t know how can a novice programmer solve such a problem if he/she has never heard about Segment Trees before, so, this is what I mean when I say that these guidelines are being “broaden” a bit too much.

Regarding the editorials, well, I happen to also have written some editorials for some Lunchtime contests and all I can say is that, it’s a very hard job to be done by a single person.

It’s advised on each contest preparation that the setters send some relevant information to the editorialist in order to make the editorials cleaner and better.

As far as I can tell, in most contests such thing doesn’t happen at all and the only time I did that was for the problem CLMBSTAIRS and everyone said the editorial was well-written.

Well, it turns out that I wrote that editorial almost fully.

So, what I would suggest to improve editorials is:

Let each problem setter write the editorial for their own problem and, if you want, add 20 dollars compensation for that, so as to acknowledge the extra work of the setter. (I say 20, because you would normally pay 200 to the editorialist for the full set, so, it is a fair division)

Regarding the correctness and clarity of the statements I guess that nothing can be done at least on a more practical level. I think so because most setters (myself included) are not native English speakers so it’s normal to have some mistakes and misunderstandings in the statements.

Maybe the testers should also proof-read statements in some way…

Thanks for having written this out though, @RandomUsername, it’s good to know that people cherish and care about CodeChef enough to write such long complaint…

Best,

Bruno

[randomusername]

@kuruma, thanks for the detailed answer. Regarding the balance of the problems, I agree with you. It’s a tough job and you pointed out some valid difficulties a team faces when creating a contest. Let’s leave that as it is for now, I would like to concentrate on other problems I mentioned. They are more important than balance.

I don’t know how payments in the world of setters/testers/editorialists work, and I don’t wanna get involved in that, but isn’t it natural for setter to write a editorial? Why would you possibly want a third person to write it on their own? I agree with you. Setter should write it, alone or in conjunction with other people. Giving it to a third person makes no sense.

As for the statements, don’t you think that 1 person, fluent in English, can fix the errors in all 10 problems relatively fast? I’m talking about just 1 person, and I’m sure you can even find ten volunteers and make them proof-read problem statements and fix errors. That can last a day, tops, and it is very valuable.

I will use the same example, the “Chef And Codes” problem in this month’s challenge. I don’t know who was the setter, and in order to remain unbiased I will not look. Just read that statement once. This is not just about English language, there are many ambiguities which make solving an easy problem quite difficult. Yet again, few people and few hours can solve it all. It’s a shame that professional coders are unable to deliver a correct problem statement, how hard can it be? There are hundreds of people on CC better than me in programming && english, and I think even I could’ve done it better.

P.S. There is a task in NOV CHALL which mentions Alexandra, Sasha and Sashen’ka, or something like that. They are all the same person. How is that even close to professionalism?

Thanks again for your effort,
RandomUsername

[kuruma]

Hello again @randomusername,

I was in a bit of a hurry when I wrote that answer and I couldn’t wrap it up properly. Fortunately, you brought it up again with the line: “(…)It’s a tough job and you pointed out some valid difficulties a team faces when creating a contest. Let’s leave that as it is for now(…)”

Maybe we shouldn’t leave that as it is and clear up some points.

First of all, most problem setters aren’t professionals, instead, they are simple ordinary people like you and me (I am studying CS, pursuing a Bachelor’s degree and I am currently on the 2nd year of a 3-year timespan), which have their own lives and their own affairs besides CC.

With this being said, I wouldn’t be surprised if you were a better programmer than I am (you probably are, because, as I’ve mentioned on earlier answers, this is the end of my first more serious year and a half learning with Codechef, and there are yet many, many topics I don’t know about, so, we are all here to learn, all the time, especially because 1 year and a half is nothing!! ), so, I guess writing problems/editorials has to be something one is really interested in, in order to do a good job.

However, as I said above, sometimes our own personal affairs, university, exams, etc, tend to be very overhelming and CC might fall back at times. It is something that ideally shouldn’t happen, but, the fact is, it does happen. That might be one of the reasons why some setters’ only check their e-mails when there are only 2/3 days left for the contest to start, which, obviously will reflect on the quality of the test cases and problem statement and puts additional pressure on the editorialist/testers’.

This isn’t something that we can fix, right? I hope people understand this. It’s not like we can SSH into the setters’ computer and leave him/her a message to check their e-mail because they are falling back on the problem preparation. It’s the inherent human factor that unbalances the equation that we can’t obviously fix (we could always discard the people which didn’t checked in after say, 1 day and replace them with someone else, but, that’s unfair on one hand and not very pratical on the other hand, as the “replacement” setter would be caught off-guard, in a sense.)

However, ironically, we have literally the best programmers’ in the world contributing for Codechef, which for sure helps in setting some great problems which can’t be found anywhere in the web, after all, which website can brag about having had @mugurelionut, @ACRush, @Anton_Lunyov, @Gennady.Korotchevic, @xiaodao, @Rubanenko, among many other amazing names setting problems? I bet no site can

So, as you can see, we have amazing setters’, somewhat competent testers and well, some poorer organization when it comes to writing editorials.

I guess that, if you agree with my idea of having the setter writing their own problem editorial, you might also agree with the point that some setters might be capable of setting a hard problem, but, might not be so capable of writing a decent editorial about it. (I really think that writing good texts takes a lot of effort and a lot of English proficiency + time, which, imho, is not something many people among the setters’ panel might have, at least, I don’t have any of those things if you ask me, which would make this task harder for some peole and might make them back off setting problems.)

On other hand, we might need to look into the problem structure versus problem difficulty versus contest duration.

When setters’ here mark a problem as being a HARD problem, it really means that the problem really is VERY HARD, right? (Which means that we are only having issues with all the remaining difficulty levels except with the HARD one, maybe simply because there are fewer harder problems and they really stand out from the rest)

However, maybe even the definition of HARD might need to be scaled down a bit, after all, if some coders here are learning Segment Trees for the first time, how is it expected that they will understand what a FFT is, how it relates with DFT, and what it is used for? We aren’t even talking about hard data-structures anymore, we are talking about concepts which are at the level of a Master’s Degree in Algorithms or maybe only PhD level in some countries (as in Portugal, for instance).

This, however, only comes into light of what was discussed earlier, and that both the concepts used and the difficulty levels need to be definitely re-thinked in order to have more balanced contests.(this is extremely notory on the Lunchtime contests, btw).

(Sidenote: I am yet to understand how to code a FFT in C++, and even understanding what it receives as input, and returns as output is somewhat tricky to me, and well, I guess I can’t call myself a newbie anymore.)

So, what can be done was already discussed above in general guidelines, we might need to wait for some admin reply on this.

Best,

Bruno

[paulrishav93]

By Day 7 or 8 of the contest, CodeChef can invite editorials from the top 10 or 20 fastest problem solvers for each contest problem. I am sure that some of them comfortable in written English, would be willing to write an editorial and in the process, we can get to learn various methods used by top programmers. However, this method will not work for the shorter contests.

[randomusername]

I see. I understand that all of you have personal lifes, but the fact that nothing will change is devastating. How can 5 day rating update/challenge problem regrade be normal? But OK, I guess it can’t be helped. Also, I still believe the problem clarity issue can be solved very easily. You didn’t talk about it as far as I can tell (I read your post yesterday). Regarding the other things, including balance and editorials, I mostly agree. You presented some valid arguments, it’s not easy as it seems. But still, I think that some things can be corrected with just a bit more effort.

As you said, there’s nothing we can say after this. Let’s just wait for an admin response, if it ever happens.

Regards,

RandomUsername

[kuruma]

Hello again,

First of all, thanks for being so insistent on this matter and for not leaving any “loose ends” in your analysis of the main issues our platform is currently facing… It’s a motive of both joy and concern to see such dedicated users

And well, I did briefly mentioned this issue regarding statements clarity, on my first post:

"Regarding the correctness and clarity of the statements I guess that nothing can be done at least on a more practical level. I think so because most setters (myself included) are not native English speakers so it’s normal to have some mistakes and misunderstandings in the statements.

Maybe the testers should also proof-read statements in some way…"

However, it might be worth mentioning that (at least, in theory) it’s the editorialist duty to revise the statements in order to make them clear!!

This brings up a more nasty problem, because, if the editorialists themselves are not native English speakers, then, there might always be some issues with problem statements and unfortunately, this is a very recurring and serious issue on some very popular websites (just look at Codeforces and at the last round which was held!! Possibly the worst round I’ve ever seen so far.) and I really don’t think that nothing can be done besides hiring maybe a good literature person to proof-read the statements

Which is also bad, because these people are getting paid and should possibly find a way to solve all these issues in a proper way!!

Please all note that I’m not being critic in a negative way, I’m just exposing some problems and I’ve already presented a plethora of possible solutions!!

@randomusername, once again, thanks! I’ll get in touch with admins and mention this post, so they can decide in an informed way!!

All the best,

Bruno

[xellos0]

I am not a native English speaker, either, yet I feel that my English is much better than the one used in some statements. (Wrong forms of verbs, order of words and lack of articles are the most common and painful issues.) That’s not an excuse, really.

If you’re not good at a language, it’s because you’re not using it regularly. One noteworthy method of improving that (as long as you don’t have the opportunity to use English at a daily basis) is reading books - that way, you get used to the correct way of using some constructions and extend your vocabulary.

Maybe a more serious problem than the level of English itself is the inclarity of statements, regardless of the language. It’s good to make a problem statement interesting by adding a story, but it should provide a clear (and that includes scientifically worded) description of the task, even for someone who decides to skip the story completely. Properly formatting sections of a statement (story, problem, input, output, notes…) and things like equations or code is another step towards greater clarity.

[hatim009]

Everything was fine earlier but the problem started after change in the rule of judging challenge problem…So I think it would be better if the codechef team will remove the rule of partial judging
and They have already reduced the number of submissions one can make for a problem.

[garakchy]

i hope everybody is familiar with Joel Spolsky(founder of stackoverflow stuff) he has GOLDEN advice for CS students(since, here we all learn coding, then we are also in that set IMHO): Advice for Computer Science College Students – Joel on Software

****> Learn how to write before graduating.

Would Linux have succeeded if Linus
Torvalds hadn’t evangelized it? As
brilliant a hacker as he is, it was
Linus’s ability to convey his ideas in
written English via email and mailing
lists that made Linux attract a
worldwide brigade of volunteers.

Have you heard of the latest fad,
Extreme Programming? Well, without
getting into what I think about XP,
the reason you’ve heard of it is
because it is being promoted by people
who are very gifted writers and
speakers.

Even on the small scale, when you look
at any programming organization, the
programmers with the most power and
influence are the ones who can write
and speak in English clearly,
convincingly, and comfortably. Also it
helps to be tall, but you can’t do
anything about that.

The difference between a tolerable
programmer and a great programmer is
not how many programming languages
they know, and it’s not whether they
prefer Python or Java. It’s whether
they can communicate their ideas. By
persuading other people, they get
leverage. By writing clear comments
and technical specs, they let other
programmers understand their code,
which means other programmers can use
and work with their code instead of
rewriting it. Absent this, their code
is worthless. By writing clear
technical documentation for end users,
they allow people to figure out what
their code is supposed to do, which is
the only way those users can see the
value in their code. There’s a lot of
wonderful, useful code buried on
sourceforge somewhere that nobody uses
because it was created by programmers
who don’t write very well (or don’t
write at all), and so nobody knows
what they’ve done and their brilliant
code languishes.

I won’t hire a programmer unless they
can write, and write well, in English.
If you can write, wherever you get
hired, you’ll soon find that you’re
getting asked to write the
specifications and that means you’re
already leveraging your influence and
getting noticed by management.

Most colleges designate certain
classes as “writing intensive,”
meaning, you have to write an awful
lot to pass them. Look for those
classes and take them! Seek out
classes in any field that have weekly
or daily written assignments.

Start a journal or weblog. The more
you write, the easier it will be, and
the easier it is to write, the more
you’ll write, in a virtuous circle.****

codechef problems are barely English, i think they are Hinglish(indian+english). sorry for that, but pls adminstrative staff, pay attn to those details. thnx

[willpun]

I was planning to write an essay about how much trouble I have gone through just to understand every single problem this site contains. But, well, forget it. It’s not worth the time ranting.
I am not a programmer but I am planning to get into the industry, switching careers. The most concern I have now is that, do you programmers always face this kind of hard-to-understand instruction to the problem every day you work? This site is scaring me due to the fact that almost every problem I’ve seen is not written in modern English!
80% of time I spent on each problem is to trying to figure out what in the holy world is it asking for!

[kcahdog]

I felt your earlier title was not appropriate so i changed it.You can change it back if you disagree.

[xellos0]

kcahdog: What do you have against Hamlet?

[randomusername]

Seriously, I feel offended. What’s wrong with Hamlet? If I broke a codechef rule, it’s ok to change what I wrote. But if I didn’t please let others decide if they like something or not.

[randomusername]

I was just about to say that. You can’t blame it all on your lack of language skills, there are mistakes that go beyond the language barrier, like in the tasks I mentioned earlier in this post.

[garakchy]

totally agree. (like

[randomusername]

Partial judging can stay. They should just judge all the submissions on 100% of test cases and then just show the score for 20%, thus when the contest ends the only thing you have to do is display results for 100%, simple as that. No regrading time.

P.S. If you don’t judge every submission on 100% as soon as it arrives, how do you know it won’t give wrong answer on some of the unknown 80%?

[randomusername]

Exactly. And I bet my rating two people who know English good can fix all the mistakes Hinglish caused in one long contest. You can pay them, but I bet you can also find volunteers. That’s why I still find it hard to believe that the best Codechef can do is Hinglish, set aside everything Bruno said. It is a hard job, and they aren’t native speakers, but we’re on internet for god’s sake, how hard can it be?

[kuruma]

I’ve already indicated this post to Admins, so, maybe in a not so distant future we might get some kind of feedback on this matter