Imbalance of Difficulty level in January Challenge 2018

jan18
shubhiks
#1

So, January Challenge ended and I noticed that:

People having 500 points solving five questions fully got rank 1846 and those having 556 points got 274 rank. Now, these class of people were able to solve problems of same difficulty yet there is a significant difference between their ranks which isn’t justified. If somebody went through the whole problem set, he would straight away figure out that 5th problem whose solution can be reduced to simple GeeksforGeeks problem would get too many submissions and the “Monsters” requiring sqrt decomposition (or other methods) won’t.

So, I have two suggestions:

  1. Make the problemset more balanced. Though obviously its not being suggested for the first time but it can become better if few people analyze the whole problemset.

  2. Remove points for meaningless brute forces. Like Why is that even there!? Run a loop from 1 to n and get 10 points, or maybe 5 points. Don’t tell me purpose is to teach people to run a loop. Its completely meaningful if the solver has thought something or its some kind of unoptimized solution. But points for completely meaningless brutes doesn’t make any sense, just for the sake of keeping it there.

Let me know your suggestions and I apologize if I sounded a bit harsh.

#2

Most of my concerns regarding your “meaningless brute” are already voiced by @swetankmodi and @dwij28 . Thank you for that :). Just adding to that, those brute force are for those new coders, especially the 2 stars so that they can fetch some points. It might be meaningless to you, but most of the times I saw them to be implementing brute Djikstras (talking about a previous long), or longest subarray sum, or some standard algorithm, which is very appreciable. Even the straight loop and performing conditions are not cakewalk for people learning basics of coding, so I think we should let this principle remain.

Regarding your example, if that guy got 554 points, then obviously he did more than meaningless brute force, else everyone else would also be 554 points. That, puts most of your arguments at rest regarding this.

I agree that the problem set was not balanced at all, for both reasons- first five being too easy and last five being too tough. There were quite a few issues with this contest, like in the challenge problem, and I hope the setting panel will make sure this doesnt occur again.

#3

@swetankmodi @shubhiks @meooow @vijju123 I don’t agree with neither of you. I really think this unbalance is unfair, but it’s not to people not doing partial scores. They could have done it easily and don’t lose, or even WIN CONSIDERABLE AMOUNT OF RATING.

And that’s what is unfair.

To those who solved more than 5 problems, instead of doing only partial scores. They can be roughly the same place as the people solving 5 and partials (which was pretty easy to be honest) and because of (IMO) weird rating system of CodeChef it will really mess up rating balance.

#4

However i think this was one of the best longs considering the questions(dp,binary search,suffix tree) (especially monster).

This is my opinion.

#5

Why aren’t the ratings updated yet?

#6

Hello everyone,

Being the setter of XYHUMOQ, I personally believe that above all reasons for having brute force sub-tasks, I would like to go with the one that’s left unnoticed in this whole conversation.

Long challenges are made to teach the participants the patience to tackle with enormous WAs or TLEs before getting that final AC which in turn makes him/her mature.

There is no doubt in the fact that brute force sub-tasks are set for beginners for them to at least get the gist of the question by solving for them on their first encounter.

But, the small or brute force sub-tasks help in a really different way.

It can help even the best programmers out there to firstly confirm that they went in the right direction and somehow the code needs optimisation for larger sub-tasks.

But, moreover, it can help you to partially verify the code that you have crafted for the larger sub-tasks. One can easily check the new code that he/she has developed by the help of the smaller sub-task because you know, you have the right solution for that and you can easily verify the new code by matching the intended outputs for the smaller sub-tasks.
By this you will at least know where the bug is if you by chance encounter one.
And personally, I myself have used this in many questions to find my fault. The most recent I remember is SMARKET from APRIL17.

And that for XYHUMOQ, the 15 point task was not that easy for first timers, it involved a good use of DP to calculate the answer for each possible string. And, since it had the X<=1000, one could have easily verified the solution for the larger task by running the code over manually created sub-tasks with X<=1000 and find the bug.

I hope I cleared the point.

Happy Bug Finding !!!

>_<

#7

If someone really cares for his rank he will definitely go through whole set.
If someone does something, then he definitely deserves better than one, who is lazy to do anything in 10 days.
Like 6 points this long or say 4 points in May’17 long, each point is earned.

#8

YES,i totally agree with @dhirajfx3 @vijju123 on earning points by doing subtasks.
it has following benifits…
1–> Better utilisation of time.
2–> keeping the hope alive for a tough one,
3–> subtasks motivate’s the new comers as well to fight for best.
4–> it keeps u in touch of the hard problem and motivate u to effort more.
5–> sometimes subtasks can teach u some usefull tricks,
for example XYHUMOQ problem , it has a very good subtasks of 15 points(not so simple at all…very few was able to find the way to count number of humangous subsequences.
but i would still say that this contest was having unbalanced problems,but having subtasks is not an issue.

#9

I feel that in Long Challenges your Rank might not be that important, as people with same marks are being ranked same. Here, I think you’re given a chance to think on each problem and as said by @include_sajal.

These contest are really helpful for newbies as they get enough time to fight on the problem.

One thing more I want to add that those who are very good programmers need not to worry about their rank in this contest, as they are stars in contest of short duration.

#10

“But, the small or brute force sub-tasks help in a really different way.
It can help even the best programmers out there to firstly confirm that they went in the right direction and somehow the code needs optimisation for larger sub-tasks.”

@include_sajal - You were spot on! In fact, in XYHUMOQ itself, solving the first subtask correctly helped me to be sure of my calculation for the no. of humongous subs. in the string. I’m not saying its impossible to do it manually, but we always trust that Green Tick more than ourselves, don’t we :smiley:

#11

This long, i felt like after seeing the problems, that problem MONSTER, KILLKTH and XYHUMOQ were of almost same difficulty level - the difficulty level you would expect for 7.5th problem (harder than usual 7th, but not upto 8th). And yes, there was a huge difficulty gap between 5th and 6th gap.

But the thing that surprised me most was the results of challenge problem. Apart from 3 users getting 100 points and 1 user getting 66.67 points, no other user even crossed 10 points.

I would voice that the presence of brute force partials is favorable, therefore, should not be removed.

#12

i agree with @taran_1407 … if any one cares so much about his rank then he should solve all 10 problems and must secure 100 points in challenge problem
what about those who are new and just trying to learn…
according to @shubhiks question 1 of codechef should start with square root decomposition and further questions should be like even gennady will find it difficult to solve…
kabhi doosron ka bhi soocho bhai

#13

Totally stucked in MONSTER after 10 points even shorten range of x which actually need to be processed to 131072(2^17) and precomputed all x from 0 to 131072 with given n still stucked.and HUMONGOUS QUERY is good question use of dp to compute subsequences of sequences and then flip. I computed sequences in optimal time but flipping create hurdle then finally i have to use brutforce for flipping and satisfy in 15 only without that dp sequences finding 15 is also not possible.i have one complain only some brutforce solution of killkth give me 5 marks in java but 20 marks in python using list comprehension…(maybe because i found that inbuilt substring method in java take linear time instead of constant time like in python).

#14

Am I the only one who’s enjoying reading everyone’s opinion!!

( ͡° ͜ʖ ͡°)

#15

humongous query, sqrgood did not have meaningless brutes. They required a lot of thinking. I solved 6 problems but couldn’t even ‘meaningless brute’ sqrgood. And yes, I know how to run a loop, still got 0 in that.

#16

2^20 not passing in Humongous query?

#17

Agree with @swetankmodi here, the meaningless brute you are referring to only holds true for 15 marks, 10 Monster + 5 Killkth. Also, you can think of marks in sub-problems as an incentive for not giving up and trying, and yeah the incentive should be relatively low, and 15 marks isn’t too much.

#18

it passed. I had a hard time coding it since i started in the last few days. I mean to they they were not meaningless. I do agree with the difference in 5th and 6th problem. it was a huge gap.

#19

Also, if everyone does meaningless brutes, then won’t the rank remain same provided everyone is able to do it since it’s very easy right?
if every 500 scorer goes to 556 by just running loops, what’s the problem? :stuck_out_tongue: (just a thought, maybe i’m wrong, but I think the brutes were fair enough this time but yeah +1 for 5th/6th gap)

#20

@dwij28 - For KILLKTH, it’s 20 points in python. I was surprised we can store string of size 10^9 in python! o_O