I want to search for strings inside a large number of documents. I have a predefined list of strings available that I want to find in each document. Each document contains a header at the beginning followed by the text and in the header are additional strings I want to search for in the text below the header.
On each iteration of document, is it possible to add the header strings after creating the initial tree that was made from the main list? Or modify the original data structure to include the new strings?
If this is not practical to do, is there an alternative search method that would be more appropriate?
If each document has its own set of strings to search for, it seems like you could just build one global Aho-Corasick matcher and then a second, per-document matcher. Then, as you process the characters in the document, feed each into both of the matching automata and report all matches found this way. That eliminates the need to add new strings to the master automaton and to remove them when you're done. Plus, the slowdown should be pretty minimal.
Hope this helps!