[SOLVED] difflib sequence matcher missing common substrings

difflib sequence matcher missing common substrings

In an attempt to find common substrings between two strings, SequenceMatcher does not return all expected common substrings.

s1 = '++%2F%2F+Prints+%22Hello%2C+World%22+to+the+terminal+window.%0A++++++++System.out.pr%29%3B%0A++++%7D%0A%7D%0ASample+program%0Apublic+static+voclass+id+main%28String%5B%5D+args%29+'
s2 = 'gs%29+%7B%0A++++++++%2F'
# The common substring are '+%', '%0A++++++++', '%s' and 'gs%29+'
# but 'gs%29+' is not matched.

import difflib as d

seqmatch = d.SequenceMatcher(None,s1,s2)
matches = seqmatch.get_matching_blocks()

for match in matches:
    apos, bpos, matchlen = match
    print(s1[apos:apos+matchlen])

Output:

+%
%0A++++++++
%2

"gs%29+" is a common substring between s1 and s2, but it is not found by SequenceMatcher.

Am I missing something?

Thanks

Solution

Perhaps the junk characters have confused the algorithm. I added a lambda function for isjunk within SequenceMatcher()

s1 = '++%2F%2F+Prints+%22Hello%2C+World%22+to+the+terminal+window.%0A++++++++System.out.pr%29%3B%0A++++%7D%0A%7D%0ASample+program%0Apublic+static+voclass+id+main%28String%5B%5D+args%29+'
s2 = 'gs%29+%7B%0A++++++++%2F'
# The expected substring is 'gs%29+'

import difflib as d

seqmatch = d.SequenceMatcher(lambda x: x in "+", s1, s2)
matches = seqmatch.get_matching_blocks()

for match in matches:
    apos, bpos, matchlen = match
    print(s1[apos:apos+matchlen])

The output is now

gs%29+