Why Codechef does not make contest problems test cases public after contest gets over ?

if someone is passing 90% of testcase, thenafter he wiil be eligible to see the testcases which are failing.
i think this is better idea.

1 Like

@l_returns it is very much similar to share your code during contest and in that case we can’t do much,
so after contest you can choose if you are willing to do it by yourselves or you want testcase,

see we can not make it as perfect as it should be ,every funtionality have some bugs all we can reduce it,yes we can implement it with some modification so that these thing can be solved, and after contest user is on his own.

@nitish235 if some of the input files are having bug then it is impossible to pass 90% testcases in that case we will never get the testcases ,and at the same time it is very difficult to decide that how much % your code should get passed to be eligible for testcases.

This issue has been raised multiple times and it was always agreed that making test cases public will only reduce the efforts of user to solve the problem rather than increasing the skill to find the corner cases. After giving a lot of attempt on a problem without getting success due to some corner case, when the user sees the actual input causing WA, instead of solving for the general corner case, he usually handles that specific scenario and go ahead and often gets WA subsequently which does not solve the problem of helping user learn to find the error.

I was also not able to get WRKWAYS accepted just due to one case and I used one accepted solution to find the scenario. I wrote a test case generator and compared the output of accepted and my solution on that and could find the unhandled case in no time.

1 Like

Stressing your solution helps in 95% cases.

1 Like

@admin @vijju123 If you are considering this I have a suggestion, how about introduce something like buying test cases using discuss karma (ofcourse after contests get over). Since discuss karma now don’t fetch you goodies, it would be good to introduce them as an exchange of test cases. Even if people start mining karma (though I doubt they would) It won’t do much harm in any case.

I would take sides with @admin in this debate.

The only case when releasing the test cases is useful is when your code is failing in small test cases which you can verify by going through your code and debug.

One of the important skill I learnt from the course Algorithmic Toolbox from Coursera is stress testing .

The basic idea is that for problems where you can easily create brute force solutions (which are guaranteed to work for small inputs) and want to debug your faster approach . You generate test cases and send it both these and compare the output.

I have personally used it to debug a lot of problems in long contests.

this is a question :slight_smile:
Please somebody explain how to generate INPUT TEST CASES FILE(in.txt) and how this is different from custom inputs,and own generated simple test cases using loops and adjustments.
Yes hemant_dhanuka rightly said you stole my words… this is exactly what i want to convey to admin please please make test cases public after the contest is over…since sept17 when is started coding i can think about number of good problems which i lefted just because there was no way lefted to do them even after attempting them so hard… editorials helps a lot they give us a more or less same or different approach which makes us learn new things sometimes but this question remain unanswered “what is wrong with this approach…what is…” and asking this question on forums is silly since how can anybody give time to your code as much as you have given and you understands it better than the reader… there are some points about why test cases should be reavealed like russian website codeforces
(1). when somebody have tried a particular approach and have synchronised each part of code seperately and days have been spent on that.
(2). somebody may not have good friends(coders) around him in colleges so no source of help except website.
my request to good/experienced coders please advise me

2 Likes

While I think keeping the judging test cases confidential is appropriate, there is a case for having some more challenging small test cases (if applicable to the problem) in editorials, or perhaps even directly in the practice version of the problem. Often the sample test cases given don’t do much more than test input and output formats.

I also like the idea of reporting test case results as happens in the %age pass problems - it’s a little frustrating to “go back” to a one-state report after problems where you get a table of outputs. Some kind of indication of the scale of the test case might make this even more useful.

1 Like

Yaa really this would help alot in many ways like it would save our time, lessen the questions posted on discuss, would make us aware of different type of corner cases and more. and it would be after the contest is over, so there will be no problem I too request @admin to add this feature

For WA, yes I can understand. But how will it help for TLE? You wont be possibly inspecting the huge arrays of size {10}^{6} to analyze your solution? Is there some tool or something which helps here?

@vijju123 No bro, i don’t have any tool. In “TWOFL” problem my all test cases were passed out less than 2.3 sec but 3 cases is not passing even in 8 sec… i am doubtfull that may be in my code have any infinite loop… so if i have test cases than i can checkout wheather i am geting ans in 10 min/100 min/…(it means TLE) or i am not getting answer after waiting for long than for sure any infinity loop is running

@vijju123 and let this TLE case go,do u think making test cases public will be very helpfull?

@pshishod2645 bro, No one “wants to” post code on forums to find where code is failing but after trying to certain level(giving 200% percent) when one think now he cant do anything further, its good to ask on discuss forum to know whats wrong in previous approach rather than skipping that question or trying other approach

2 Likes

No its not infinite loops xD. Since each file only has a single TC, its very possible to get large variations over time. I faced that too. For me, TC 1 of task-3 was failing. Had to optimize my code :frowning:

@pshishod2645 its not compulsory to look at testCases… if one wants to try more he/she can try more without looking at test cases

1 Like

Making test cases public, well, to be honest small cases can help. It was an idea for my future editorials to keep a testcase bank (couldnt do it in ltime60 due to lack of time to analyze solutions).

We cannot force codechef to reveal them, and more often than not they are maximal cases with N=1000 in smaller ones and N={10}^{5} in larger ones.

@vijju123, What if the editorial includes a link to the test files? The test files will hardly exceed 6MB in size. Once downloaded, it’ll be upto the users whether they want to wait until “Notepad” responds, or they want to keep trying to find out the corner case without it…XD

2 Likes

@sarthakmanna for me its good Idea

He is talking about making them public after contest so the analogy of ACM ICPC doesn’t hold valid. Besides, in codeforces you can see the testcase for which your code fails after the contest and it is very helpful in practice. The suggestion to do the same on codechef is a very valid one.

1 Like