pythonhashnearest-neighborlocality-sensitive-hashbigdata

Confusion in hashing used by LSH


enter image description here

Matrix M is the signatures matrix, which is produced via Minhashing of the actual data, has documents as columns and words as rows. So a column represents a document.

Now it says that every stripe (b in number, r in length) has its columns hashed, so that a column falls in a bucket. If two columns fall in the same bucket, for >= 1 stripes, then they are potentially similar.

So that means that I should create b hashtables and find b independent hash functions? Or just one is enough and every stripe sends its columns to the same collections of buckets (but wouldn't this cancel the stripes)?

Would a dictionary be enough for a hashtable in this case*?

*Is a Python dictionary an example of a hash table?


Solution

  • I think I figured it out, posting for future readers.

    I am going to use one dictionary, since the slides mentioned that it's OK to use the same hash function for every stripe (dictionaries do that).

    Every bucket will be a key for our dictionary.

    On insertion, a document (i.e. a column which belongs in a stripe) will be passed by a hash function (which we will create) and the result should be a key. That way our dictionary will be populated.