For example:
// file foo
def my_func():
print("foo!")
// file bar
class Bar:
def __init__(self):
pass
def hello():
import foo.my_func
my_func()
import dill
dill.dump(Bar(), open("bar.dpkl",mode='wb+'))
$: python3 -m venv foobar
$: source activate foobar/bin/activate
(foobar) $: python3
>>> import dill
>>> obj = dill.load(open("bar.dpkl",mode='rb')) # just fine
>>> obj.hello()
foo!
But, suppose I throw in a twist:
class Bar:
def __init__(self):
import numpy
Now,
>>> obj = dll.load(open("bar.dpkl",mode='rb'))
ModuleNotFoundError: No module named 'numpy'
It appears as if the linked C
-binaries are not bundled up in the pickle
file. However, if the code is genuine, native python
code, the pickle
file output by dill
is essentially a dynamic archive.
Is there any way to extend this dynamic archive behavior to include the libraries like numpy
et al.? Or must these sort of C
+ python
-shell libraries always be installed in the executing python
environment?
Basically, I think the thing I am asking for is a pythonic version of a jar
that can be generated straight from python
code: is this possible? Or is a container with requisite dependencies the jar
I am looking for?
A possible solution:
import dill
class Bar:
def __init__(self, foo):
self.basename = foo.name
self.deps = deepcopy(foo.deps)
self.foo = '-'.join([self.basename,'foo'])
self.bar = '-'.join([self.basename,'bar'])
def install(self):
import pip
pip.main(["install"] + self.deps) # this method will be deprecated soon
def load(self):
self.install()
foo = dill.load(open(self.foo,'rb'))
return foo
def store_self():
with open(self.bar + '.dpkl','wb+') as out:
dill.dump(out,self)
@classmethod
def store(cls, foo):
bar = cls(foo)
bar.store_self()
bar.install()
with open(self.foo + '.dpkl','wb+') as out:
dill.dump(out,foo)
class Foo:
def __init__(self,name,*arg,**kwargs):
self.name = name
self.deps = [ "numpy" ]
def hello():
import numpy
print("foo")
...
foo = Foo("Hello! I'm a foo!")
Bar.store(foo)
$: python3 -m venv venv-foo
$: source venv-foo/bin/activate
$: python3 -m pip install dill
>>> import dill
>>> bar = dill.load(open('bar.dpkl','rb'))
>>> foo = bar.load()
>>> foo.hello()
Hello! I am a foo!
No warranty of copy-pasteability, but the gist of it is there. Might have to tinker with it a bit to get the Bar.dpkl
working. It is also sensitive to python3
's version.
With some more work, native pickle can probably be used to store Bar
, which can then be used to install dill
.