ANSLEAK Scoring

general
1

According to the ANSLEAK question,

During the contest, the displayed score will account for exactly one test file from each group, i.e. your score reflects your submission’s performance on 33% (5/15) of the test files. However, if your program gets a non-AC verdict on any test file, your submission’s verdict will be non-AC. In other words, an AC verdict denotes that your program runs successfully on all the test files. After the end of the contest, your score will be changed to include the sum of your program’s scores over the remaining ten test files.

I’m curious and wanted to ask does non-AC mean the same as TLE here? I am asking because my hill climbing solution that has passed 5/15 test files in the allotted time has no points increased.

I am guessing that It got TLE while being assessed for the remaining 10/15 Cases.

Whereas the number of points that my initial mode finding approach increased a lot.

For reference after complete evaluation,

My better solution that got me 87 points during contest now gets me 31 points
https://www.codechef.com/viewsolution/31820914

The naive mode finding approach that got me 74 points during contest now gets me 78.6
https://www.codechef.com/viewsolution/31227406

If the reason why my 87 points solution didn’t increase for other test cases was TLE, may I implore the @admin to consider not putting a time constraint on presently accepted solutions.

Thanks for your precious time everyone!

2

Non-AC includes all other verdicts - WA, TLE, RE, etc. Though the point made in the paragraph you’ve quoted had been changed mid-way through the contest as was explained in one of the announcements:

7th April 15:53 IST: Due to increasing submission queue, and large time limits, we have modified the Challenge problem (ANSLEAK). From now onwards, your submissions will be evaluated only on the 5 ‘live’ test files. The 10 ‘non-live’ test files will be added after the contest, and all submissions will be rejudged on all the 15 test files again to get the final score.

This has two impacts:

  1. If your solution somehow gets an AC on all the 5 live test files, but gets a non-AC on the remaining 10, during the contest you will get an AC verdict, but after the rejudge that would change to a non-AC verdict when it is evaluated on the remaining 10 files.

  2. If you have a randomized solution without a constant seed, your score on the 5 live test files could change after the rejudge.

And, as the latest announcement mentions, the rejudge of the challenge problem is still underway and not yet over. Wait till we announce that it’s over to compare the scores.

1 Like
3

Thanks for informing about the same @admin. The scores of many users who have fake - solved are much higher than the ones who should have got a better score as of now.