MATTEG - Editorial

Problem Link

Practice
Contest

Author: Alexandru Valeanu
Tester: Hasan Jaddouh
Editorialist: Bhuvnesh Jain

Difficulty

MEDIUM

Prerequisites

Recursion, Bitmask Dynamic Programming, Hashing

Problem

You are given a grid, R*C, where each location has some number written on it. You are also given N teleportation pairs (dx, dy), meaning that you can go from (a, b) to (e, f) iff |a-e| = dx and |b-f| = dy, where |x| denotes the absolute value of x. You are required to find a sequence of steps so that you use as maximum number of teleportation pairs (atmost N), each one only once, and try to maximise the sum of the number on the locations you visited.

Quick Explanation

Since, we need to use each teleportation pair once, we can store a bitmask where the set bits denotes which what teleportation pairs are still not used up. Then we can simply iterate over the possible next states based on the current location and bitmask and use dynamic programming to solve the above problem. Since, the number of locations that might be visited are limited, try to optimise the dynamic programming solution using hashing to handle the case of excess memory usage.

Explanation

Subtask - 1

No dynmaic programming or any advanced techniques are required to solve this subtask. Only simple recursion which traverses all the possible positions depending on the remaining teleportation pairs is enough. But this solution is quite slow to pass other subtasks.

Subtask - 2

We can simply see that subproblems of many recursive calls of above solution are overlapping. To clearing differentiaite between the different states, we create an dynamic programming array consisiting of 3 states, namely the current x-coordinate, current y-coordinate and the mask denoting the teleportation pairs still to be used. Thus our dynamic programming array looks like dp[x][y][mask].

Building the above the dynamic programming bottom up, would lead to complexity of O(R*C*N*{2}^{N}), which is quite slow for final 2 subtasks. Also, the memory used by the above approach will be of the order of O(R*C*{2}^{N}), which would cause memory limit issues with last 2 subtasks as well.

Subtask - 3

In the above approach it is easy to see that most of the states would be unreachable. For example, there might not exist any sequence of steps (of length <= N), such that we can reach from (a, b) to (c, d). Also, there might be cases we could reach from (a, b) to (c, d) only through particular sequence of moves, meaning that all “mask” positions might not be visited also. Thus, we need to avoid these redundant calculations in the above approach and also try to find an more strict upper-bound on the number of visited states in the above dynmaic programming solution.

Let us first define what a valid teleportation is. A teleporation is said to be valid if from a location on a grid, we can go to a location present on the grid. It is invalid, if we go to a location outside the grid.

We try to find a tighter upperbound for the number of states visited. Now, let us say, we are at a particular location, (a, b), then we have 5 possible choices to take from the current location, i.e. either stay in the current location (since we can’t use any valid teleporation) or jump in any of the 4 directions (if jump is valid i.e. inside the grid). Also, at each steps, we would have to choose from K of the remaining teleportation pairs to go to next point. Thus, we can say that we have an upperbound of O(min(N*{5}^{N}, R*C*N*2^{N})) on the number of states that are visited. (Why minimum is taken is because, we either visit all the states in which case the bound is simply R*C*N*{2}^{N} or we can’t take some valid teleportation pair at some location and visit less states, leading to N*{5}^{N} bound).

Now, let us try to write the above solution in top-down appraoch i.e. using recursion with memoisation. This, will lead us to visiting only those states which are reachable from starting state i.e dp[Sx][Sy][{2}^{N}-1]. Thus, in recursive solution for subtask - 1, we simply add the dynamic programming approach. But there is still a catch here i.e the memory requirement. The memory required by this solution is still O(R*C*{2}^{N}), which limits us from passing this subtask. But our time complexity of the solution has decreased. To get a trade of between the time and space complexity, we can use “maps” or “unordered maps” to store the list of possible “masks” or “locations” as per implementation. This will increase the time complexity by logarithmic factor, but the space complexity would now be order of the number of possible states. Using this optimisation, we can easily pass the given subtask.

Subtask - 4

We are almost close to the full solution now. We just need to lower the trade off we did between memory and time in the last approach. For this, we will use hashing approach, to remove the logarithmic factor from the above last approach. (It may seem that unordered_map would suffice for this subtask too, but since they number of states in the unordered map are large, rehash would be called many times, which is a costly operation and lead to the solution becoming slower).

First let us consider mapping 3 states into a single number without any collision at all. Since, (a, b) are grid points, we can represent each grid point into a single integer as a * C + b. Thus now, the states reduce to 2. We can again apply similar operation to reduce the states to 1. For example mapping (a, b, mask) to single integer (a * C + b) * {2}^{N} + mask. The order of this number is O(R*C*{2}^{N}). We can simply hash this number to another value, i.e. build a hash table for these values. Suppose, we have successfully build our hash table, then we can easily see that we can retrieve the elements in O(1), thus removing the logarithmic factor from above solution. Care must be taken while choosing the size of hash table. Taking it too small would be of no use due to large number of collisions. We should chose the size around the number of states that can be reached, thus reducing the number of collisions while inserting the keys into it. Also, we can either implement hashing using linear probing or separate chaining. The author chose linear-probing to implement the hashing technique.

For more details, you can refer to the author’s solution below.

Tester’s Solution for full problem

The tester’s solution used the fact that we have 5 choices at each and every point on the grid. Thus similar to bitmask where we have 2 bits to represent a states, he represented each states used base 5 encoding where 0 means we stay at the same location, and 1 to 4 means we chose one of the 4 directions. Simliar to iterative bitmasking solution which are written traversing each mask from 0 to 2^N - 1, the solution traverse each base 5 mask from 0 to 5^N - 1. Then based on the values in the mask, we just perform the teleportation and check whether we land at valid grid point or not. If yes, we update the answer for that mask. The final answer is the maximum over all the mask values calculated. The time complexity of this solution is exactly N * 5^N, and it doesn’t use any hashing. Also, the 2 dimensions where we store the current x and y coordinate in above solution is not required here as it is captured in the base case, where we store the cost of starting point.

For more details, you can refer to tester’s solution below.

Time Complexity

O(min(N*{5}^N, R*C*N*{2}^{N}))

Space Complexity

Size of Hash Table ~ O(min({5}^N, R*C*{2}^{N}))

Solution Links

Setter’s solution
Tester’s solution
Editorialist solution - For 70 points, (No implementation of hash table)

8 Likes

I used brute force to check all cases, and terminated the search if we went outside of the grid, or it was sure if we can’t get better sum than our previously found best. (Added the max value in the grid to the current sum x times, where x is the number of not used tel-pairs)

I got 100 points for this approach, which quite surprised me, because it’s possible that these terminations don’t occour oftenly enough, and then the complexity is O(4^N*N!*T) = O(4^9*9*5) = 4.75*10^{11}.

Maybe the tests aren’t strong enough.

3 Likes

The practice link is incorrect.

Can anybody tell me why my brute force solution gives WA for test case 1?
Link

Submissions CodeChef: Practical coding for everyone and CodeChef: Practical coding for everyone are identical, still the later gave a WA during the contest which led to a penalty. Please check the issue.

can anyone please explain how this (N * (5 ^N)) comes and hoe it reduces the dp states (how does skipping some possible position leads to correct answer) ?

According to the editorial at each step we have 5 choices to move and k choices to choose from the available teleportation pairs so at each step the shouldn’t the number of possible ways to move equal to 5k . Since there are n steps the total no of ways should be (5k)^n where k<=n . So this must be equivalent to (5n)^n instead of n*(5^n).Correct me where i am wrong.

What is the significance of multiplying with 2^N in (a∗R+b)∗2^N+mask?

I tried multiple approaches with and without unordered_map, nested unordered_map, multiplying with 2^N(as mentioned), reserving memory in advance, changing the max_load_factor and what not but nothing seems to work.

I always get a TLE on Task#3 of Subtask#4.

Is it really that necessary to implement a custom hash table. Some insights are really appreciated.

PS: Nice question.

Thank You :slight_smile:

I wrote a simple recursion for the first subtask in Python 2.7, and it’s working for all the given test cases, but is giving WA when I submit. Amy boundary conditions or logic gaps that I’m missing in my code?

https://www.codechef.com/viewplaintext/15129394

Can someone please point out the error in my solution?

I submitted first solution, using @likecs 's idea, without hashing and got 70 points. Then, I tried the solution using hashing which fails at 3-4 test cases.

Link 1: Without hashing. Gets 70 points.

Link 2: With hashing. Fails on 3-4 tests.

@likecs I didn’t get how using DP from “0 => (2^N-1)” to “(2^N-1) => 0” changes the time complexity. Can you provide some resources for reading about it. Also I don’t get why dp[R][C][2^N-1] will be a trouble. It can be made easily as we need only ints and max 2GB space will be taken up. Help would be appreciated.

@kingofnumbers

Can anyone explain me how does the tester’s solution account for different permutations of teleportation that may be applied.

Please Clarify.

PS: i thought of the same approach as tester’s but failed to account for permutations of teleportation used.

Thanks in advance

I suspect a bug in the setters solution.
I tried running the setters solution on this test case:

1
3 5 2
1 2
1 0
2 1
1 1 1 2 1
1 1 3 1 1
4 5 1 1 1

The correct answer should be 12 (given by 3 + 4 + 5), but I actually get the answer 9.

The line

auto code = ((x * (R - 1) + y) << 10) ^ mask;

should read

auto code = ((x * C + y) << 10) ^ mask;

I know! I have seen your solution during the contest. You are not the only one you managed you get full points by using this idea. First of all, congratulations! I didn’t think of this idea!

Also, I would like to apologize to all contestants who got full marks using the ‘right’ solution(s).

2 Likes

Updated. :slight_smile:

No penalty in lunchtime :slight_smile:

I can’t see a WA associated with the faulty submission.

@alexvalenau Will there be more test cases in practice? I feel a little guilty after seeing the much harder solution here, and knowing my solution could be easily countered by some test cases.

I will add some tomorrow. You should not get 100p after that!

You don’t have to feel guilty! I would have done the same (and I am sure I am not the only one).

2 Likes

Please help!