I have a dbm file that leads to an error when I try to access a value directly:
>>> import dbm
>>> db = dbm.open('scrapers','c')
>>> key1 = db.keys()[0]
>>> db[key1]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
dbm.error
>>> db.get(key1)
'(dp1\nS\'username\'\np2\nNsS\'password\'...etc...
It only happens with this file, it contains pickled classes of scrapers to various websites.
I can't provide the actual db data because it contains login data. My question is: what are possible explanations for this problem? Where should I start looking to debug this?
If you need any other info (except for the data itself) please let me know.
Turns out there's a detail to dbm that isn't explained in the python dbm docs:
The sum of the sizes of a key/content pair must not exceed the internal block size (normally between 512 and 4096 bytes).
I'm using gdbm now, and this works just fine.