Regarding Problem Setting

I’d like to know the answers to a few questions.

How many maximum days/months/years does it take for problem idea acceptance/rejection?
Being a member of Codechef, do I have the right to at least receive one line rejection of my problem idea in days/months/years?
Is Codechef biased on choosing questions based on the author’s previous number of questions?

My Experience:

The first problem I tried submitting was based on DFS + DP. I wrote a considerable solution to at least receive any sort of email. After not receiving any remark, I doubted if the problem was good and unique enough. I let that go after a month. I sent a few more good problems. Again, no remarks. Finally, I sent a decent unique question which was at least worth a reply. I was determined to receive at least one email and hence I mailed them for any remarks/suggestions. After numerous emails over a span of 1-2 months I received an email mentioning “you’ll soon receive remarks”. I took that as a positive sign and designed another decent problem. Little did I know that it was the last email I’ll ever receive from Codechef on remarks.

Then something happened which got me thinking if it’s safe to submit ideas on Codechef. Fortunately, my doubt dissolved when I got to know the real reason. My exact problem idea was asked in one of the short contests. I was furious as I had submitted the same idea months before. After numerous emails to support/help Codechef, I got to know the situation. The problem which I sent was supposed to be reviewed by Long challenge admin and the other similar problem was in one of the 6 problems submitted in a set, reviewed by another admin. He submitted late, got selected. I submitted early, no verdict. Okay, shit happens. I didn’t bother much then.

I submitted another decent problem, hoping for a last chance.
I sent an email, received no replies. It’s okay, they might be a bit busy.
I sent another email, received no replies. It’s okay, they might be busy.
I sent another email, received no replies. It’s okay, they might be seriously busy.
I sent another email, received no replies.

It’s okay. Life’s tough.

PS: I still love Codechef.

15 Likes

I applied 4 months before yet not received any reply from there side…so you can wait until they reply

4 Likes

One line soln for all of your problems.

13 Likes

Haha this is funny ! :laughing: I was also unaware about CC traditions, but now learning slowly

2 Likes

I’d not mind to wait. Infact, That’s what I’ve been doing. Just, if Codechef has a maximum time limit in which a person is guarranted to recieve reply, even if it’s 6 months, i’ll wait with the certainity factor. Assuming remaining life to be around 60 years, 120 emails are sufficient to set one problem on Codechef.

1 Like

That post is hilarious :joy:

1 Like

I’ve got a couple of Problems burning a hole in my pocket - not sure if they’re good enough for here, though, though I think they would have done well at hackerrank before they changed the rules about what they’d accept - so this (and the other thread) are very informative - thanks for posting :slight_smile:

4 Likes

Really? That’s awesome! I’m sure they’re gonna be quality questions. I’d love to take a look. :nerd_face:

4 Likes

Hey, I think so they check the tags for the problem, like they cannot accept multiple problems on the same topic(they do filter out only the better one).
In my case I heard from them just within few hours, so its not like that they take months or years to get back to you.

P.S. : Another problem of mine is still in the queue for 2-3 months(so yeah I guess they have rejected my idea).

1 Like

Actually - is it possible to submit a Problem that goes straight into Practice (i.e. not for a Contest, and so is, presumably, unpaid?). I posed one in another thread that looked like a minor variation of an existing one but actually required a fairly interesting solution, but I’ve blabbed about it on the forums now so it’s definitely not suitable for a Contest XD

1 Like

Scratch that: I came up with a (hopefully) better one :slight_smile:

I do have some questions, though; I’m very much hoping that someone with some knowledge of the process can answer them:

  1. Does the problem page support the same Markdown + Latex etc as the forums? (see How should the problem statements be formatted on Codechef, but it is 5 years out of date)
  2. Does it support Footnotes?
  3. When providing the Problem, are details of the approach kept secret from Testers? (Want them to have a blank slate)
    3.1 There’s one Problem that I have a somewhat involved solution for, but am worried that there’s some “trick” to solving it much more simply. Can I eschew the “problem approach”?
    3.2 What about details of the test plan?
  4. I’m hearing lots of people simply don’t get a response to their Problems.
    4.1 Does the NDA still apply? i.e. my problem is “lost” and can’t be submitted anywhere?
    4.2 For people that got explicit rejection - what were the reasons? Were they fair?
  5. How good are the checks that the Problem is original?
    5.1 We’ve had a couple of unoriginal Problems slip through the net in the last two Long Challenges, but how many get caught before “going live”?
    5.2 I’m very paranoid about this, especially as I’ve not really looked at many Problems outside of Hackerrank - accidentally submitting a duplicate and having to withdraw it must be mortifying :frowning:
    5.3 Plus, there’s the fact that once you’ve finally solved your own Problem, it suddenly seems really “obvious” and you think that surely, someone else must have come up with it at some point XD
  6. If all tests are supposed to be generated automatically (using the Setter-submitted testcase generator, presumably with some input from the Tester), then why are there so many formatting issues in testfiles?
3 Likes

They replied in just not more than a month!!

Hey, do you still need answers to these questions? :slight_smile:

1 Like

I don’t need to know anymore, but am curious about the answers :slight_smile:

2 Likes

Fair. :slight_smile:

  1. Yes, problem page supports latex now. Earlier it was pure html but it has been revamped pretty long ago.
  2. I don’t think it supports footnotes (if the type of footnotes you are referring are same as those at end of page in word docs or books). But you can have a notes section and afaik latex can be used to control text size to some degree so it might help perhaps?
  3. Yes. Only the contest admin has the solutions. These are given to tester on his request (most of those rare times were for those super hard problems). If you want, you can request contest admin to make tester have an independent solution as well as attempt the problem himself (at least conceptually). But you need to provide the solution to contest admin - that helps in selection of problems sometimes as many problems are rejected for impossible solutions. Test plan details are like, you just state how you made the test cases and what corner cases you considered. That helps tester and admin have an overview and let the tester suggest improvements.
  4. You can apply for the problem elsewhere - at least thats what I have heard happening. It makes sense to me as well, because afaik you shouldn’t submit your proposal elsewhere after its confirmed accepted. Before that I don’t think they mind it. Tagging @admin for a formal response.
  5. A lot. Mostly its the experience of contest admin. You know, some problems you’d know on looking at them that “Its not possible for such an idea to be unique. A lot of people must’ve thought of it before as well.” . They try to ascertain that duplicate problems are rejected in first stage itself. I feel its very rare where neither the tester nor the admin are able to find out a problem already existing. They do use google search as well, btw.
  6. Usually happens when setters are new. They forget to run Codechef’s format check code on their test cases after uploading, mess up like not giving a new line after last input (messes up python/JAVA codes) or sometimes they just hand make a TC and violate the format. They are primarily due to being negligent and not following the required protocol because they are new.
4 Likes

Excellent; many thanks for the detailed and informative response :slight_smile:

Edit:

A couple of further questions, if I may:

  1. On Hackerrank, the Author can access a (private) simulated view of the Problem, and so submit solutions and time how long it takes on Codechef’s servers, etc, and also see exactly how it will be rendered when viewed by a contestant - is that kind of thing available here?

  2. Hackerrank also has a rigid set of rules regarding the naming of test input files and of output files. Is Codechef the same, or is it more free-form? Does this naming scheme (if any) including which subtask the test input belongs to (or maybe there is a folder for each subtask and the relevant input files go inside it)?

2 Likes

I edited my answer to answer all the points. Realized that pressing tab automatically posts the thing XD

1 Like

Thanks! I edited my “thankyou” post with a couple more questions XD

1 Like
  1. Yes. You upload the problems at http://campus.codechef.com (account there is made by admin only for setting panel). A testing page is created for you, which is more or less like the contest. There you can add your problem and see how it is looking. Its slightly better when hosted on codechef, but it does the job. You can upload test data there, submit solutions and everything necessary.
  2. No rigid rules. In fact, test file name isnt even visible. However, for purposes of automating the uploading of TC, it is recommended that you rename input files with a .in extension, output files with a .out extension, and keep the names of corresponding input-output files same. If you follow that, you can use the automated script for uploading test cases. Else you’d have to do it by hand, which again, is not a very major No, you MUST follow the rule but more or less like a minor inconvenience to you depending on how many test files you have (and their size). (Codechef does not have the zip upload feature as of now :frowning: , so we use a script to upload test files XD)
2 Likes

Great; thanks again - I think I’ll skip this Long Challenge and work on getting these polished up and submitted :slight_smile:

Oh - will it count against me if I submit (say) three Problems in rapid succession? They are intended for Long Contests, and I judge them to be (respectively) Medium, Easy-Medium and Medium-Hard.

Oh - and are “sequel” Problems that are thematically similar to existing Problems (but which require completely different approaches to solve them) discouraged?

1 Like