NOV13 - Feedback

challenge
feedback
november

#1

This thread is to discuss the overall experience of the November Challenge 2013. Feel free to express your views/feedback.


#2

we have end sem exams in our colg and so cudnt spend much tym on november challenge… although the number of submissions for the problem PRETNUM has gone quite high now, it was a nice question for me to solve, implementing the segmented sieve for the first tym… i really wish i cud have spent a little more tym on CHEFGM… it was a really nice problem… what was disheartening though was that the five easy questions all had over a 1000 submissions, CHEFGM had 308, and the rest of the 3 problems 23, 21 and 9… it would have been better to have questions with gradual progressive difficulty… i really wish to be a part of a contest in which the number of submissions would be roughly as follows (1 being the easiest):

  1. 3000 (a beginners question)
  2. 1000 (easy implementation)
  3. 700-800 (medium adhoc implementation)
  4. 400-600
  5. 200-400
  6. 100-200
  7. 50-100
  8. 20-50
  9. 1-10

of course, its just my point of view… many of u wud obvsly differ…


#3

I had a bad experience with one question in this November challenge.The question Yet Another Nice Girl took me a whole day to find an efficient way to solve it.It took me a lot of time to derive the formula and compute an efficient way since the constraints were huge…My solution involved not only ,Segmented Sieve but also Prime factorization to find the perfect square and hence I had to be extra careful with optimisations… It took me a whole day.
However. I was really put off when one of my associate just put up a few ist numbers of the sequence and after putting it in OEIS.com instantly came to know the formula i.e p^(q-1) where both p and q are prime numbers.This is not fair at all.Please its an earnest request to not put such questions whose formula can be easily found in OEIS.


#4

@deepai_dutta i too spent more than a day on that problem… maybe that associate of urs submitted it in pretty quick tym… but i guess u must have had a lot of satisfaction after having solved it… i can say for myself that i was happy…


#5

1)Both the problems Superpowers of 2 and Missing some chairs were more or less based on fast exponentiation. I think problems should be from different areas as it helps to gain exposure to more topics.

2)The easy questions were too easy. There were 4 problems with almost 2000 submissions or more. I think there should be only around 2 and the other 2 problems should be around 1000-1500 submissions.

3)Problem Statements:
a)the problem statement for Chef Game had one very important piece of information missing. It should have been explicitly mentioned that “all numbers in the pile are unique”. b) The problem statement for Sereja and vectors was very difficult to understand and poorly worded. I hope more effort is put in framing of problems from next contest.

  1. If possible there should be another problem like Chef game with around 200-300 submissions. Most people got stuck on 6 problems as the problems ahead were very tough.

  2. Something should be done about comment moderation during the contest. Some user called mojojojo7 had put up link to a solution for Pretnum on a chinese website. Although it was deleted by @bruno_adm quickly, a fair number of users might have had access to it. Hope that was not the reason for the increase in submissions for pretnum towards the end!!

  3. There is a lot of confusion about the rejudging process for the challenge problem. I think the process should be explained or atleast we should be updated regularly about the current state of rejudging and when the process will be complete.

Overall it was a decent contest and Pretnum and Chefgame were really nice questions.


#6

Very easy and very tough problems.

There is no moderate problems … Two easy problems should be replaced with moderate problems.
Most of the contestants solved 5 problems … While these contestants have huge difference between each other.


#7

I liked the problem set, as I do every month, but as mentioned by @predator0 the problems were kinda “one sided”. I am using codechef for over a year now and it’s a wonderful site with lots of support, editorials, a good community and a lot of diverse algorithm problems but there are some issues that seem to be increasing contest after contest.

TEST CASES: In certain problems the test cases seem to be sort of weak.

Most users have experienced the consequences of not addressing this issue properly. If it’s caught early during the contest the submissions are re-judged and of course some solutions fall and it can be a frustrating experience for obvious reasons. If it’s caught close to the end of the contest, re-judging stops being an option because this approach doesn’t give enough time for the users to strike back but it still doesn’t feel right because wrong solutions were able to pass. What I suggest should be done is to put a little more work into generating test cases or to implement a system similar to what is used in codeforces and topcoder which is challenging submissions after the end of the contest, the later being my favorite option.

PROBLEM STATEMENTS: Some problem statements are kind of complicated to understand and unpleasant to read not just because of mathematical reasons and a few ones don’t have enough specifications.

A large part of the community doesn’t have English as their native language (that includes me) but since it is by far the language most used in IT it only makes sense to use it as a means of communication among the community. Of course grammar mistakes are expected but that doesn’t serve as an excuse to neglect in writing problem statements, grammar mistakes are not the biggest issue here. Some problems lack specifications which feels like an handicap to users trying to solve them. This is a very minor issue because it’s very rare and can usually be safely corrected during the contests but still I mentioned it just to be out there and on the mind of the problem setters.

COMMENT SECTION: There are some useful comments but sometimes it feels like a place of crying for help, providing test cases or to brag.

This issue bothers me a lot, even more than cheating and I’ll explain why in a moment. First of all, the comment section is to provide some feedback, regarding something that can’t be understood in the problem statement or some weird behaviour in the evaluation of submissions and it has been useful to correct some mistakes or to add important specifications in the problem statement itself but often it becomes a debate place and this is not entirely users fault. Usually, the users that post such comments are unfamiliar with the rules and/or don’t realise what they’re doing. What I don’t understand is why this is more strictly addressed in the cook offs. One of the rules in the cook offs is that admin will only publish the comment if the comment is appropriate, why can’t the same be done in long contests? I mean, it would make even more sense to apply this in the long contest rather than the cook offs. A cook off usually lasts 2 hours, give or take. A long contest usually lasts 10 days. Inappropriate comments can cause a lot more damage in 10 days. And since in every contest users get to “debate” the problem in the comment section it is becoming acceptable. This annoys me more than cheating because cheaters choose to do so, they are aware of the rules but choose to cheat, it’s not the community or admins fault but the comment section is supposed to be controlled to prevent the usual behaviour. I’ve often seen users provide hints right after solving the problem. That just takes the fun out of solving the problem, by providing hints we’re just preventing the other ones to learn something by themselves and we’re being unjust to the users who have already solved the problem. I believe that’s mostly to brag, showing that you solved the problem and somehow you’re the one who can help other users solve it too.

================================

So far I’ve only made critics but don’t get me wrong, I love codechef, the problems, the community, I truly think is an amazing place to connect with other programmers, to learn interesting stuff and to compete. I’m not trying to point fingers here and I don’t have any kind of hatred feeling towards anyone, everybody makes mistakes. The problem setters are doing an amazing job, I couldn’t dream that there were so many different problems with so many approaches, the little I know about algorithms I mostly owe it to codechef. I’m aware that the issues mentioned above are not easily solvable for numerous reasons: lack of people, lack of time and so on… I’m not an expert but I’m not useless either so I could provide help in assisting with the 3 issues mentioned. The reason I don’t want do so is because I would hate to miss a contest, since helping with the issues mentioned would give me an unfair advantage over the other competitors during the contest. As I said before I owe a lot to codechef, so I could eventually “sacrifice” a couple of contests to help out with what I can. I’m not trying to offend anyone, really, and nothing I’ve written here is final so if you disagree with me feel free to downvote but please comment as to why and give me a chance to either change my point of view or to provide a good argument to defend it. Oh and I know this thread is about NOV13 but I had to generalise because those things happen almost every contest.


#8
  1. with the confusion surrounding the rejudging of challenge problem,
    serious thoughts must be put into this regard, looks like judging scheme was changed too soon, seems to be a bad decision specially right now.

  2. Regarding the question set there was wide difference in difficulty level of problems,
    ques GERALD2, MONOPLOY, QPOINT got less than 25 AC solutions each
    ques CHEFGM got 300 odd AC submissions.
    while other problems each got > 1000 AC solutions.
    What I want to say is that, there could have been more problems in CHEFGM difficulty level.(MEDIUM)
    Nothing needs to be said for challenge problem, it has got issues of its own :frowning:


#9

When I started with Codechef, I don’t think questions (barring the first one or two) were this easy. In Nov’13, any beginner could solve atleast 4 questions, if not 5, quite easily and that too within the span of one or two days. This is not what the Long contest is meant for. If this be it, then better organize multiple short contests rather than a Long contest because it has started to become a bit boring. The first five questions (based on difficulty) are always based on implementation or Number Theory.
It would not be wrong to say that the first five questions (based on difficulty) could have easily appeared in a short contest and yet be solved by almost everyone who has basic knowledge of Number Theory.
That being said, I always look up to the Codechef Long contest. But it’s time the difficulty level is increased for the first few questions.

This is what I suggest as the pattern for Long contest: 1 cakewalk problem, 1 easy problem, 2 easy-medium problems, 2 medium problems, 1 medium-difficult problem, 2 difficult problems and of course, 1 challenge problem.

Another suggestion: Please show personal rank (Global and Country) on the contest page itself. We need to either search our name on contest ranking page (which is again messed up as it takes time of submission into account) or keep clicking on Next until we reach our name on the contest page.


#10

Hello @all,

From my side, I can only be sorry due to the fact that the problems were too similar, but as I’ve mentioned to some people, they were devised with some months time difference, and, as such, I was completely unaware that they could be used together on the same contest.

MCHAIRS was actually written not even thinking about the binary exponentiation method, as I had seen its original derivation formula as a modifcation of the Binomial Theorem, so, I really didn’t thought about it much.

SPOTWO was intended to be an easy problem and I wrote my solution being mostly concerned with the implementation of a long arithmetic idea to “avoid” overflow of the number 2*N_bin… (I knew that around the number 524 thousand and something or 527 thousand and something, the number 2*N_bin would exceed the value of an ULL and that was the only reason I used 600.000 as an upper limit - again - I didn’t even realized the problem could be so easily solved as almost everyone did).

On the other hand, if I had found some way of failing the most naive solutions and use only the idea that Hiroto told me about (which is largely based on Fermat’s Little Theorem), then this problem would have been MEDIUM and not EASY.

Besides this small issue with my own problems, the only really big issue I have to point to the contests, is the way the tester’s and/or setter’s tend to classify the problem difficulty levels.

For instance, we are told, while setting problems, to send an e-mail with our thought difficulty level and the problem requisites, possibly even BEFORE the problem is selected.

This is not good.

This is not good because a problem I can see as being MEDIUM or HARD can easily be seen as SIMPLE or even CAKEWALK by the given month’s tester.

Since the problems must be previously selected by taking into account the difficulty level we thought about (at least on an early stage), this opens room for a lot of ambiguity on the problem difficulty level classification.

I think that if we can have a better problem selection method based on the problem difficulty, than we can have more balanced contests…

(This will also eventually matter for the Tester him/herself, as if a tester is more experienced than other, he/she can have different points of view on the difficulty levels)

Best,

Bruno


#11

One issue I have with the hard problems is that two of them, GERALD2 and MONOPLOY, are nearly identical in terms of solving technique, so basically the same people solved them. Not only it creates an unfair advantage for people well-versed in that technique, but it provides less opportunities to learn new stuff, which is my personal reason for participating in the contests.


#12
  1. One think that was really irritating was slow website throughout the contest (and still is, took me 5 minutes to sign in to write this comment). For a 10 days long it should make little difference; but as someone pointed out it is a busy month for students because of the upcoming exams and course projects, setting aside time to solve some problems only to find the site hanging is a mood killer.

  2. Many questions were raised regarding why scores were not visible for challenge problem. No admin answered them :frowning: That coupled with 1) above was too much to take.

  3. Question quality has been dwindling since MAY13. JUNE13 had too easy questions. JULY13 was comparatively good, but after that all contests had sharp difference between easy and tough problems. Some more problems on medium level would be nice.


#13

the only thing i want to improve is the way editorials are explained.they must be written keeping in mind about naive users. taking case of chefgm editorials:this editorial what i think was not written
keeping in mind that there might be some users who don’t have any knowledge about game theory(specifically no mention was there regarding what is partial or impartial game)…


#14

Sadly, russian translations were not of good quality. With all the respect to translators, it might be done better. Russian translation doesn’t sound like a Russian language, more like a machine translation. Also there are some errors in translation, for example ‘logistic’ was translated into Russian as ‘logic’ and was making no sense in Russian version. Also I don’t understand why name “Sashen’ka” is not translated in Russian, it looks ugly being simply copied from English version as is.


#15

I agree with everything @junior94 said, especially with “TEST CASES: In certain problems the test cases seem to be sort of weak.”

This happens in almost every long contest: that there is a problem with weak test cases, sometimes even many problems in one contest.

It’s quite unfair to users implementing a proper solution that will handle the given constraints only to find out that other users used much simpler solutions which would fail in certain specially-crafted inputs.

It’s also sad for those other users because they will never know the real difficulty of the problem, and thus will never learn new algorithms/techniques. It’s even worse because they will think that their solutions are correct or are fast enough even though they are not, just because the CodeChef judge ACed their submissions.

I think the team should consider making more exhaustive and powerful test cases, that will really stress-test the submissions in terms of time limit, corner cases, and will withstand most common greedy/heuristic solutions. This is especially important in a 10-day long contest where pretty much all kinds of greedy solutions and heuristics will appear. Please, when creating a problem, put yourself in the solver’s shoes and think of all possible greedy solutions, and try to create test cases that will break them. Also consider the naive solutions and find a way for them not to pass the time limit if they are not the intended solutions.


#16

It’s hard to see where my solution failed. Often, I feel that I have the right solution but incorrect output format. Probably some extra characters at the end. Whenever I see “internal error” or Time with -1.00 value I have no clue what happened.


#17

I dont like the partial test file rule.


#18

Yes as far as I am concerned I am really satisfied :slight_smile: got to learn really awesome things like how to do fast factorization even when numbers are greater than range of 10^6 :smiley:


#19

this is what the motto of the contest is… that u learn new things, new algos… we shouldn’t be bothered by what others do :slight_smile:


#20

@deepai_dutta that p(q-1) thing is basic combinatorics, i learnt it in +1 only… :slight_smile:
so it was pretty easy…