I'm using Happybase/python to get data from hbase and I already used some filtering, but now for some reason it won't work.
I have the filter for my scanning code goes like that :
f = "SingleColumnValueFilter ('input', '', =,'substring:{}')".format(keywork)
res = pTable.scan(reverse=True,limit=1000, filter= f.encode('utf-8'))
Which btw works Now this inputcolumn has records like this :
"{'institution_id': '023', 'application_id': '01', 'channel_id...}"
the problem is that if I set keyword to lets say:
keywork = "instition_id"
it works, but obviously that's not what I need, so when concatinate it with the value I'm looking for :
keywork = "instition_id': '"+ my_value
I get thriftpy.transport.TTransportException: TTransportException(type=4, message='TSocket read 0 bytes')
After new tests I conclude that the problem is with the single quotes, but I don't know why or how to fix it.
I tried keywork = "instition_id\': \'", keywork = 'instition_id\': \'' none of that worked. It's probably some thing silly but it driving me crazy.
And I'm thinking about regex, maybe I'll give it something like institution_id?: ?my_value with ? to tell it any character would do fine but I don't know nothing about regex so ...
Good thing you thought about regex, I don't know why it doesn't work even with escaping the quotes (I hope someone who knows would explain that to us), but here is something to get you going:
f = "SingleColumnValueFilter ('input', '', =,'regexstring:.*institution_id.: ." + your_value + ".*')"
the . means any character so the quotes won't trigger an error this time.
I'm also a beginner with regex so I expect anyone to refine, edit that line.