PROBLEM LINKSDIFFICULTYMEDIUM PREREQUISITESString Matching, AhoCorasick, Dynamic Programming PROBLEMYou are given a dictionary D, of W words. You are given a set of N strings, S, to search for the frequency of occurance of the W words. EXPLANATIONThis is a classical problem in multiple string searching. If the task gave a single word, then using KnuthMorrisPratt would have yeilded linear time complexity. But, repeating the same approach for each word will lead to O(summa(strings in D) + W * summa(strings in S)) computations, which is too much for the time limit. The indended solution requires the use of the AhoCorasick string matching algorithm. The idea is that you build a data structure for all the given set of words in O(summa(strings in D)) computations. This can then be used to search for each occurance of any of the words in the strings in S. This is primarily an extension of the KnuthMorrisPratt structure's failure function, where you allow transitions across different strings on failure  called the suffix arc in the wikipedia article. The wikipedia article is quite informative and contains further links to help you with implementing this datastructure, including, links to implementation in C. The time complexity of using the AhoCorasick algorithm to find each match is O(summa(strings in D) + summa(strings in S) + summa(frequency of each word)). The sum of the frequencies of the words is quite large. count the frequency of wordsThus, we cannot look for individual matches and add them up. We do a clever dynamic programming on the AhoCorasick data structure.
AhoCorasick iterates through the matches by backtracking through the data structure while following only the dictionary suffix arcs. Since they lead to matched dictionary words, the complete backtrack prints every match. The data structure, along with the dictionary suffix arcs, creates a directed acyclic graph which is expected to be traversed to print each match. Since we don't want to print each match
exampleLet us say that there are two words
The AhoCorasick data structure looks like There is one dictionary suffix arc in light green. We process the string acabbab. When we have processed the first 4 characters, the standard algorithm expects to print cab and then follow the dictionary suffix arc and print ab. But, we only store a count of 1 on the node b that is reached at cab. We refrain from backtracking at this point. We continue and eventually store 1 when we end up on the node b that is reached at ab. Later, we follow the links and add up the count values by visiting the graph bottomup in breadth first manner. This means that we add the 1 at the b in cab to the 1 at the b in ab because they hßave a dictionary suffix arc between them. Refer to the tester's solution for details. SETTER'S SOLUTIONCan be found here. TESTER'S SOLUTIONCan be found here.
This question is marked "community wiki".
asked 12 Aug '13, 16:01

I tried using AhoCorasick algorithm but got TLE , perhaps because I didn't optimize it . Suffix trees and KnuthMorrisPratt also gave me TLE . Finally I did it using Suffix Arrays and it got accepted . answered 12 Aug '13, 17:18
5
@vineetpaliwal : it seems you have used DC3 algorithm for suffix array generation.could you pls write idea for this algorithm.i am pretty sure many more people along with me will get benifited.
(12 Aug '13, 18:11)

Well , suffix array solution is : 1 . build suffix array in O(N) // EDITED 2 . for each pattern use binary search in sorted suffix array for find first occurance in suffix and last occurance in suffix then solution for this pattern is last  first + 1 . 3 . here is implementation answered 12 Aug '13, 17:20
I tried that approach but it timed out.
(12 Aug '13, 18:06)
it's DC3 algorithm for suffix array construction and only need o(n) to build.
(12 Aug '13, 18:13)
It teased me that even DC3 gave TLE n I didn't know y.
(12 Aug '13, 18:41)
yes zeulb is right implementation is O(n) but O(nlogn) is enough when n = 5,000,000 = 100 * 50,000 .
(12 Aug '13, 19:34)
I also got AC using suffix array. O(n log n) suffix array construction timed out for me but O(n) DC3 algorithm worked.
(12 Aug '13, 20:20)
can you give me a link to your code please?
(13 Aug '13, 02:35)
I also got AC with O(n log n) suffix array. The important observation for me is to create a suffix array for each of the 100 strings, instead of a single suffix array for their concatenation. If not, I get TLE.
(15 Aug '13, 01:54)
showing 5 of 7
show all

can't it be done by purely suffix tree (w/o bsearch etc) ?? http://bit.ly/163JdyG (3rd slide) answered 12 Aug '13, 16:58

i implemented boyre moore algo .. and i was getting wa .. i dont know on which test case is my solution failing .. pl give me a test case on which it fails and ans for that test case.. http://www.codechef.com/viewsolution/2525073 thanks :) answered 13 Aug '13, 17:57

can u elaborate aho corasick lil bit more...... answered 20 Aug '13, 00:20

Can somebody, who knows c++ better explain ? http://www.codechef.com/viewsolution/6842375 http://www.codechef.com/viewsolution/6842373 "#define where _end gives" runtime error, but without Accepted. Why is that? answered 06 May '15, 05:53

@gamabunta: Just fixed the setter/tester solution links.
the practice and contest links aren't working correctly..
The practice and contest links are link LYRC
Please fix the practice and contest links to below respectively. http://www.codechef.com/submit/LYRC http://www.codechef.com/AUG13/problems/LYRC