pythonpython-2.7pickle

How to use append with pickle?


I need to append to a pickle file (as I don't have the entire dictionary with me at one go). So for doing the same I have written the following code:

 import pickle

 p = {}
 p[1] = 2
 q = {}
 q['a'] = p
 p = {}
 p[2] = 0
 pickle.dump(q, open("save12.p", "ab"))
 f = {}
 f['b'] = p
 pickle.dump(f, open("save12.p", "ab"))

However, when I am loading the pickle file I don't find the value of dictionary f there.

How should I go about appending in a pickle file?

Also, databases like 'dbm' are not working for my need, as I am working on Windows.


Solution

  • Pickle streams are entirely self-contained, and so unpickling will unpickle one object at a time.

    Therefore, to unpickle multiple streams, you should repeatedly unpickle the file until you get an EOFError:

    >>> f=open('a.p', 'wb')
    >>> pickle.dump({1:2}, f)
    >>> pickle.dump({3:4}, f)
    >>> f.close()
    >>> 
    >>> f=open('a.p', 'rb')
    >>> pickle.load(f)
    {1: 2}
    >>> pickle.load(f)
    {3: 4}
    >>> pickle.load(f)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    EOFError
    

    so your unpickle code might look like

    import pickle
    objs = []
    while 1:
        try:
            objs.append(pickle.load(f))
        except EOFError:
            break