When providing the Problem, are details of the approach kept secret from Testers? (Want them to have a blank slate)
3.1 There’s one Problem that I have a somewhat involved solution for, but am worried that there’s some “trick” to solving it much more simply. Can I eschew the “problem approach”?
3.2 What about details of the test plan?
I’m hearing lots of people simply don’t get a response to their Problems.
4.1 Does the NDA still apply? i.e. my problem is “lost” and can’t be submitted anywhere?
4.2 For people that got explicit rejection - what were the reasons? Were they fair?
How good are the checks that the Problem is original?
5.1 We’ve had a couple of unoriginal Problems slip through the net in the last two Long Challenges, but how many get caught before “going live”?
5.2 I’m very paranoid about this, especially as I’ve not really looked at many Problems outside of Hackerrank - accidentally submitting a duplicate and having to withdraw it must be mortifying
5.3 Plus, there’s the fact that once you’ve finally solved your own Problem, it suddenly seems really “obvious” and you think that surely, someone else must have come up with it at some point XD
If all tests are supposed to be generated automatically (using the Setter-submitted testcase generator, presumably with some input from the Tester), then why are there so many formatting issues in testfiles?
Yes, problem page supports latex now. Earlier it was pure html but it has been revamped pretty long ago.
I don’t think it supports footnotes (if the type of footnotes you are referring are same as those at end of page in word docs or books). But you can have a notes section and afaik latex can be used to control text size to some degree so it might help perhaps?
Yes. Only the contest admin has the solutions. These are given to tester on his request (most of those rare times were for those super hard problems). If you want, you can request contest admin to make tester have an independent solution as well as attempt the problem himself (at least conceptually). But you need to provide the solution to contest admin - that helps in selection of problems sometimes as many problems are rejected for impossible solutions. Test plan details are like, you just state how you made the test cases and what corner cases you considered. That helps tester and admin have an overview and let the tester suggest improvements.
You can apply for the problem elsewhere - at least thats what I have heard happening. It makes sense to me as well, because afaik you shouldn’t submit your proposal elsewhere after its confirmed accepted. Before that I don’t think they mind it. Tagging @admin for a formal response.
A lot. Mostly its the experience of contest admin. You know, some problems you’d know on looking at them that “Its not possible for such an idea to be unique. A lot of people must’ve thought of it before as well.” . They try to ascertain that duplicate problems are rejected in first stage itself. I feel its very rare where neither the tester nor the admin are able to find out a problem already existing. They do use google search as well, btw.
Usually happens when setters are new. They forget to run Codechef’s format check code on their test cases after uploading, mess up like not giving a new line after last input (messes up python/JAVA codes) or sometimes they just hand make a TC and violate the format. They are primarily due to being negligent and not following the required protocol because they are new.
Excellent; many thanks for the detailed and informative response
Edit:
A couple of further questions, if I may:
On Hackerrank, the Author can access a (private) simulated view of the Problem, and so submit solutions and time how long it takes on Codechef’s servers, etc, and also see exactly how it will be rendered when viewed by a contestant - is that kind of thing available here?
Hackerrank also has a rigid set of rules regarding the naming of test input files and of output files. Is Codechef the same, or is it more free-form? Does this naming scheme (if any) including which subtask the test input belongs to (or maybe there is a folder for each subtask and the relevant input files go inside it)?
Yes. You upload the problems at http://campus.codechef.com (account there is made by admin only for setting panel). A testing page is created for you, which is more or less like the contest. There you can add your problem and see how it is looking. Its slightly better when hosted on codechef, but it does the job. You can upload test data there, submit solutions and everything necessary.
No rigid rules. In fact, test file name isnt even visible. However, for purposes of automating the uploading of TC, it is recommended that you rename input files with a .in extension, output files with a .out extension, and keep the names of corresponding input-output files same. If you follow that, you can use the automated script for uploading test cases. Else you’d have to do it by hand, which again, is not a very major No, you MUST follow the rule but more or less like a minor inconvenience to you depending on how many test files you have (and their size). (Codechef does not have the zip upload feature as of now , so we use a script to upload test files XD)
Great; thanks again - I think I’ll skip this Long Challenge and work on getting these polished up and submitted
Oh - will it count against me if I submit (say) three Problems in rapid succession? They are intended for Long Contests, and I judge them to be (respectively) Medium, Easy-Medium and Medium-Hard.
Oh - and are “sequel” Problems that are thematically similar to existing Problems (but which require completely different approaches to solve them) discouraged?
Oh - will it count against me if I submit (say) three Problems in rapid succession? They are intended for Long Contests, and I judge them to be (respectively) Medium, Easy-Medium and Medium-Hard.
Not at all! In fact, they’d love active setting participation from the community. There is a never ending requirement for problems of higher difficulty!
Oh - and are “sequel” Problems that are thematically similar to existing Problems (but which require completely different approaches to solve them) discouraged?
What matters is the approach. Just to make the answer more general, I will answer for other cases as well-
If all 3 problems have similar solutions, then it makes sense to accept only one of them.
If all 3 have different solutions, but of same topic, they might not be included in same contest but thats all. This is to make sure contests are not imbalanced over a topic.
If only story is same - not an issue at all. Like, nobody cares for story at all (unless it becomes extremely big )
Ended up procrastinating a little too much, so I’ve submitted the Problem Idea P1 (of three) at Problem Idea Submission | CodeChef to light a fire under my @$$
Let’s see if it gets a response!
Edit:
Skipped P2 for now as, despite having a working implementation, the Sketch Approach isn’t fully formalized yet, but submitted problem P3
Ended up submitting P2 last Tuesday, and it’s already been approved by @alex_2oo8 - that’s 2 out of 3 so far, with a very quick turnaround; kudos to the Codechef Team