pythonpickleshelve

python shelve is not saving/loading


When I save/load my workspace via functions in a subfile, shelve doesn't work (test1). However, if I do the same in one file, it works (test2). Why is that? How can I fix the problem for the first case?

In the main file:

# in saveWS.py   

# to save
def saveSlv(fileName):
    destination='./'+fileName+'_shelve.pkl'
    bk = shelve.open(destination,'n')
    for k in dir():
        try:
            bk[k] = globals()[k]
        except Exception:
            pass
    bk.close()
        
# to restore
def loadSlv(fileName):
    myshelve ='./'+fileName+'_shelve.pkl'
    bk_restore = shelve.open(myshelve)
    for k in bk_restore:
        globals()[k] = bk_restore[k]
    bk_restore.close() 

In the main file:

import shelve
# User defined functions
from saveWS import saveSlv, loadSlv 

# It doesn't work
a=1,2,3
b='ypk'
fileName='test1'
# save the variables in work space by calling a function
saveSlv(fileName)
del a, b
# restore the work space by calling a function
loadSlv(fileName)
 
# It works
a=1,2,3
b='ypk'
fileName='test2'

# save the variables in work space
destination='./'+fileName+'_shelve.pkl'
bk = shelve.open(destination,'n')
for k in dir():
    try:
        bk[k] = globals()[k]
    except Exception:
        pass
bk.close()
del a, b

# restore the work space
myshelve ='./'+fileName+'_shelve.pkl'
bk_restore = shelve.open(myshelve)
for k in bk_restore:
    globals()[k] = bk_restore[k]
bk_restore.close()

Solution

  • Here's the globals()-related part of your problem (I guess):

    Return the dictionary implementing the current module namespace. For code within functions, this is set when the function is defined and remains the same regardless of where the function is called.

    So globals() in your functions is always the namespace of saveWS.py.

    And here the dir()-related one:

    Without arguments, return the list of names in the current local scope. With an argument, attempt to return a list of valid attributes for that object.

    Therefore dir() refers to the local namespace within the function.

    You probably could fix that by passing dir() and globals() as arguments:

    def saveSlv(fileName, variables, namespace):
        destination='./'+fileName+'_shelve.pkl'
        bk = shelve.open(destination,'n')
        for k in variables:
            try:
                bk[k] = namespace[k]
            except Exception:
                pass
        bk.close()
    
    def loadSlv(fileName, namespace):
        myshelve ='./'+fileName+'_shelve.pkl'
        bk_restore = shelve.open(myshelve)
        for k in bk_restore:
            namespace[k] = bk_restore[k]
        bk_restore.close() 
    

    Calling:

    saveSlv(fileName, dir(), globals())
    loadSlv(fileName, globals())
    

    You could do without the variables argument by using

    ...
        for k in dir(sys.modules[namespace['__name__']]):
            ...
    

    instead (after importing sys). In your usecase you could probably further replace namespace['__name__'] with '__main__'. If you want to get rid of all the additional arguments you could try if this works for you:

    import shelve
    import sys
    
    def saveSlv(fileName):
        destination='./'+fileName+'_shelve.pkl'
        bk = shelve.open(destination,'n')
        namespace = sys.modules['__main__'].__dict__
        for k in dir(sys.modules['__main__']):
            try:
                bk[k] = namespace[k]
            except Exception:
                pass
        bk.close()
    
    
    def loadSlv(fileName):
        namespace = sys.modules['__main__'].__dict__
        myshelve ='./'+fileName+'_shelve.pkl'
        bk_restore = shelve.open(myshelve)
        for k in bk_restore:
            namespace[k] = bk_restore[k]
        bk_restore.close() 
    

    Check here if that's appropriate.