×

# Why Codechef does not make contest problems test cases public after contest gets over ?

 11 my humble request to @admin to share test cases of contest problems with editorial, or provide feature to see test cases during submission after contest gets over.. while practicing one get WA, TLE or whatever error it must show testCases where code is failing similar to codeforces, hackerRank, hackerEarth etc. plateforms.. Reason is simple, during practice, when we try to upsolve contest problem, it will save our time to figure out what's wrong with our code whenever we gets WA, then we try our best to figure out corner cases where code is failing, if succeed than its good but if do not gets succeed than we post link to our submission on discuss, and we have to wait someOne will tell us corner case... most of times on codechef Discuss forum, someOne helps to figure out in which test case code is failing but someTimes we do not get replies... if we know test cases, then we can figure out by ourselves where code is failing... and it will save our time to post "find where code is failing" type question on discuss and will also save others time of answering those questions.. i am facing this issue to figure out whats wrong with my june long challenge "TWOFL" problem sorry for by bad English.. asked 12 Jun '18, 21:46 533●1●12 accept rate: 3% Yaa really this would help alot in many ways like it would save our time, lessen the questions posted on discuss, would make us aware of different type of corner cases and more. and it would be after the contest is over, so there will be no problem I too request @admin to add this feature (12 Jun '18, 21:49) ssp5473★ For WA, yes I can understand. But how will it help for TLE? You wont be possibly inspecting the huge arrays of size ${10}^{6}$ to analyze your solution? Is there some tool or something which helps here? (12 Jun '18, 22:19) @vijju123 No bro, i don't have any tool. In "TWOFL" problem my all test cases were passed out less than 2.3 sec but 3 cases is not passing even in 8 sec.. i am doubtfull that may be in my code have any infinite loop.. so if i have test cases than i can checkout wheather i am geting ans in 10 min/100 min/...(it means TLE) or i am not getting answer after waiting for long than for sure any infinity loop is running (12 Jun '18, 23:02) @vijju123 and let this TLE case go,do u think making test cases public will be very helpfull? (12 Jun '18, 23:03) No its not infinite loops xD. Since each file only has a single TC, its very possible to get large variations over time. I faced that too. For me, TC 1 of task-3 was failing. Had to optimize my code :( (12 Jun '18, 23:08) Making test cases public, well, to be honest small cases can help. It was an idea for my future editorials to keep a testcase bank (couldnt do it in ltime60 due to lack of time to analyze solutions). We cannot force codechef to reveal them, and more often than not they are maximal cases with $N=1000$ in smaller ones and $N={10}^{5}$ in larger ones. (12 Jun '18, 23:33) 1 @vijju123, What if the editorial includes a link to the test files? The test files will hardly exceed 6MB in size. Once downloaded, it'll be upto the users whether they want to wait until "Notepad" responds, or they want to keep trying to find out the corner case without it...XD (12 Jun '18, 23:52) @sarthakmanna for me its good Idea (13 Jun '18, 00:05) @sarthakmanna - Its essentially revealing the test cases at the end of the day. While I agree that one can put in corner cases there, I doubt that codechef will agree to reveal all the test cases. Its something the team is very particular about. (13 Jun '18, 00:24) showing 5 of 9 show all

 4 I recommend you to not post your code on forums, asking for which corner test case your code fails. That said, figuring out the corner test cases also comes under skills which you will learn by yourself. If test cases are available to you then you won't think of them, But suppose in your ACM-ICPC regionals you know how to solve 5 problems but 4 of them are giving W.A. due to some corner test cases, what will you do then?? This skill to figure out the corner test cases will come to you only if practiced over time . Codechef is better as it is. answered 12 Jun '18, 22:44 825●1●13 accept rate: 13% 1 @pshishod2645 bro, No one "wants to" post code on forums to find where code is failing but after trying to certain level(giving 200% percent) when one think now he cant do anything further, its good to ask on discuss forum to know whats wrong in previous approach rather than skipping that question or trying other approach (12 Jun '18, 23:06) 1 @pshishod2645 its not compulsory to look at testCases.. if one wants to try more he/she can try more without looking at test cases (12 Jun '18, 23:17) He is talking about making them public after contest so the analogy of ACM ICPC doesn't hold valid. Besides, in codeforces you can see the testcase for which your code fails after the contest and it is very helpful in practice. The suggestion to do the same on codechef is a very valid one. (13 Jun '18, 00:11) dwij285★ 2 Larger ones are not revealed by CF fully. Smaller edge cases can be put up by editorialist. That can be done, but it needs analysis of what contestants are doing to figure out what can be corner cases for various approaches. (13 Jun '18, 00:25)
 4 UVA online judge has an excellent utility udebug for comparing outputs to given test cases they provide. I think if you provide the test cases someone can work to make such a utility available to people. It would be very much helpful in debugging the code answered 14 Jun '18, 13:46 51●4 accept rate: 0%

@vijju123 I think the problem setter and tester will have worked hard for getting corner cases included in a particular problem... They why to keep a load on editorialist for the same... I mean setter or tester can also give some small corner cases separately with their question for editorials... It will be a "cake-walk" thing for them as they will have spend some time for making test cases files... So by this even if editorialist is in hurry (which u were last time) then too people can get help...

1.6k211
accept rate: 23%

I agree editorialist can do that nicely.. but with help of problem setter and tester the "public test cases" can be made of excellent quality... And they ll be more useful...

(13 Jun '18, 01:19)

PS: Adding some corner test cases on editorial is a very good idea...

(13 Jun '18, 01:22)

Though I also agree with @admin to not to reveal whole test case file as finding your own mistakes is kind of "fun" and a "skill"... Initially I was also frustrated and many people even get frustrated due to not revealing test cases in every contest... But after sometime I enjoyed that...
And in fact now I have hardly any question which was not submitted and got AC even after knowing logic properly... I learnt how to write neat bug free codes... And that is a skill I aquired by codechef..

(13 Jun '18, 01:29)
1

And even I don't get TLE's due to lack of minor optimizations... If I got logic then AC is for sure.. though it may take sometime... For eg. In VSN I got many wrong answers due to lack of practice in handling large double values.. so eventually "not revealing" also helps...

# but as suggested by @vijju123 the idea to post some small corner cases in editorial is very good... And even it don't reveal the whole test cases file and spoil the fun of debugging...

(13 Jun '18, 01:32)
1

Yes, editorialist will take setter and tester's help only xD. But mostly there are cases which may or may not be "corner cases" depending on your approach. For that, analysis of solution is required.

(13 Jun '18, 14:40)
 1 Most of you are saying debugging is important skill which obviously true. But what you are forgetting is many time after hours of trying to debug you still don't get why it isn't passing. Then you just give up on the problem, you can't go on forever. Instead if you know where your code is failing then you can try to correct your code. In any case if you don't want to see the test cases you don't have to nobody is forcing you. answered 13 Jun '18, 10:59 298●1●9 accept rate: 8% That is when u need help of other solutions or ur friends or discuss... If it would be easily available then everyone will give up earlier... ik it's upto us but then too if it was easily available then why a "beginner" will go for debugging... (13 Jun '18, 12:13) Why wouldn't a beginner go for debugging? Anybody who wants to learn to solve problems in contest will try to learn it. On the other hand if we know test cases we can at least still learn the concept instead of giving up. (13 Jun '18, 14:37) The giving up one is a valid point. But more often than not, such cases are very specific to the approach you employed. A common bank may be hard to accommodate this. Regarding revealing all test cases, most of the people (especially newcomers) arent disciplined enough to know how to use that. This move will do more harm as they will have poor debugging skills leading to poor performance in contest and demotivation. (13 Jun '18, 14:39) Exactly that was my point @vijju123... They aren't disciplined enough.. (13 Jun '18, 14:53) If u give some students a "copy chit" in exam and ensure that they won't be punished then many of them will take it.. they won't think about themselves.. that's why people keep supervisor even in practice tests... (13 Jun '18, 14:56)
 1 This issue has been raised multiple times and it was always agreed that making test cases public will only reduce the efforts of user to solve the problem rather than increasing the skill to find the corner cases. After giving a lot of attempt on a problem without getting success due to some corner case, when the user sees the actual input causing WA, instead of solving for the general corner case, he usually handles that specific scenario and go ahead and often gets WA subsequently which does not solve the problem of helping user learn to find the error. I was also not able to get WRKWAYS accepted just due to one case and I used one accepted solution to find the scenario. I wrote a test case generator and compared the output of accepted and my solution on that and could find the unhandled case in no time. answered 14 Jun '18, 19:45 524●1●6●10 accept rate: 17% I totally aggred with You! (14 Jun '18, 22:34)
 1 Stressing your solution helps in 95% cases. answered 17 Jun '18, 14:40 99●3 accept rate: 0%
 0 I think not providing the test cases help us to improve our debugging skills and due to "no option" we are forced to apply different concepts and analyse our solution again and again which helps us to understand our implementation and the concept deeply. But it is only helpful when we have "no option" and still we want to get it perfect so that we give it some more try , but what happened if we have "no option" at all ?? we have to move forward while skipping that problem which is not desirable for anyone. At the same time what happened is,we get some unexpected verdicts due to wrong test input files or weak test cases which is not a fault of programmer in this case it is must to have the test cases so that we can report someone to look into the matter. The core thing which i feel for any contest is that we have "no option" during the contest, all we have to do it by ourselves which is helpful for improvement but after contest we have editorials and many more options , I think we can do something similar to the testcases we can set a number "K" after "K" number of attempts that user is eligible to have test cases anytime after the contest. answered 14 Jun '18, 16:54 4★dk30390 84●4 accept rate: 9% Then people will share test cases by making fake submission.. which isbnot desirable... U can always ask for help to people after lot of efforts from you... Many people can help u with that actually if ur soln is neat enough and u did tried properly to get answer before asking (14 Jun '18, 17:42)
 0 if someone is passing 90% of testcase, thenafter he wiil be eligible to see the testcases which are failing. i think this is better idea. answered 14 Jun '18, 18:12 96●1●6 accept rate: 33% Doesnt encourage thinking about edge cases.Also, what if lot of edge cases there? (17 Jun '18, 18:21)
 0 @l_returns it is very much similar to share your code during contest and in that case we can't do much, so after contest you can choose if you are willing to do it by yourselves or you want testcase, see we can not make it as perfect as it should be ,every funtionality have some bugs all we can reduce it,yes we can implement it with some modification so that these thing can be solved, and after contest user is on his own. answered 14 Jun '18, 18:20 4★dk30390 84●4 accept rate: 9%
 0 @nitish235 if some of the input files are having bug then it is impossible to pass 90% testcases in that case we will never get the testcases ,and at the same time it is very difficult to decide that how much % your code should get passed to be eligible for testcases. answered 14 Jun '18, 18:43 4★dk30390 84●4 accept rate: 9% The question setter or Tester provide input testcases, so I dont think testcases contains any type of bugs. (14 Jun '18, 18:56) see this https://discuss.codechef.com/questions/126761/wrong-test-input-in-stmincut?page=1#126931 it is possible to have bug in input test caes. (14 Jun '18, 21:13) dk303904★
 0 @admin @vijju123 If you are considering this I have a suggestion, how about introduce something like buying test cases using discuss karma (ofcourse after contests get over). Since discuss karma now don't fetch you goodies, it would be good to introduce them as an exchange of test cases. Even if people start mining karma (though I doubt they would) It won't do much harm in any case. answered 18 Jun '18, 02:02 537●1●8 accept rate: 27% Doesnt address main things like “Most of TC are huge and hence of no use.” Also, instead of buting using karma, the present system of asking n awarding karma is better and more sustainable :) (18 Jun '18, 02:06) Yeah, but it helps in the following cases: - 1) deciding whether solution would have passed through just optimizations or it was an edge case. It has mostly happens when I submit using python (like passing 19 out of 20 test cases). - 2) many of the codechef questions have a testcase file comprised of various test cases(which individually may not be very large to deal with). We can just find the failing case using diff of expected answer and my answer. (18 Jun '18, 02:23) Buying using karma may promote more engagement with the discuss community. Also this will help in the case where beginners won't be able to harm themselves by just looking at the test cases. There is a +ve covariance between ratings and karma to some extent. (18 Jun '18, 02:26)
 0 I would take sides with @admin in this debate. The only case when releasing the test cases is useful is when your code is failing in small test cases which you can verify by going through your code and debug. One of the important skill I learnt from the course Algorithmic Toolbox from Coursera is stress testing . The basic idea is that for problems where you can easily create brute force solutions (which are guaranteed to work for small inputs) and want to debug your faster approach . You generate test cases and send it both these and compare the output. I have personally used it to debug a lot of problems in long contests. answered 19 Jun '18, 22:20 4★bhishma 297●7 accept rate: 11%
 0 While I think keeping the judging test cases confidential is appropriate, there is a case for having some more challenging small test cases (if applicable to the problem) in editorials, or perhaps even directly in the practice version of the problem. Often the sample test cases given don't do much more than test input and output formats. I also like the idea of reporting test case results as happens in the %age pass problems - it's a little frustrating to "go back" to a one-state report after problems where you get a table of outputs. Some kind of indication of the scale of the test case might make this even more useful. answered 17 Aug '18, 21:55 5★joffan 948●8 accept rate: 13%
 toggle preview community wiki:
Preview

By Email:

Markdown Basics

• *italic* or _italic_
• **bold** or __bold__
• image?![alt text](/path/img.jpg "title")
• numbered list: 1. Foo 2. Bar
• to add a line break simply add two spaces to where you would like the new line to be.
• basic HTML tags are also supported
• mathemetical formulas in Latex between \$ symbol

Question tags:

×2,476
×1,427
×365

question asked: 12 Jun '18, 21:46

question was seen: 3,103 times

last updated: 21 Jun '18, 11:08