Note that parts of the following description are specific to the Python package mujoco_py but the issue itself is general.
I followed the steps described in here and here to built a container with mujoco-py installed in it that uses GPUs for rendering. However, when do import mujoco_py I get the following errors when mujoco-py is trying to create some files/directories that are needed for rendering with GPUs. However, it is not possible to create any files inside the container after it is built as the file system becomes read-only, except if those files are going to be stored in /tmp. Also, in my case, it is not an option to build the container as writable. I also tried installing mujoco-py via python3 -m pip install . -e instead of python3 setup.py install but got some other, similar errors when I did import mujoco_py.
Is there a way to either change the path where those files are going to be created or keep those files in memory?
import mujoco_py
running build_ext
building 'mujoco_py.cymj' extension
creating /usr/local/lib/python3.7/dist-packages/mujoco_py-2.0.2.13-py3.7.egg/mujoco_py/generated/_pyxbld_2.0.2.13_37_linuxgpuextensionbuilder
Traceback (most recent call last):
File "/usr/lib/python3.7/distutils/dir_util.py", line 70, in mkpath
os.mkdir(head, mode)
OSError: [Errno 30] Read-only file system: '/usr/local/lib/python3.7/dist-packages/mujoco_py-2.0.2.13-py3.7.egg/mujoco_py/generated/_pyxbld_2.0.2.13_37_linuxgpuextensionbuilder'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.7/dist-packages/mujoco_py-2.0.2.13-py3.7.egg/mujoco_py/__init__.py", line 3, in <module>
from mujoco_py.builder import cymj, ignore_mujoco_warnings, functions, MujocoException
File "/usr/local/lib/python3.7/dist-packages/mujoco_py-2.0.2.13-py3.7.egg/mujoco_py/builder.py", line 510, in <module>
cymj = load_cython_ext(mujoco_path)
File "/usr/local/lib/python3.7/dist-packages/mujoco_py-2.0.2.13-py3.7.egg/mujoco_py/builder.py", line 105, in load_cython_ext
cext_so_path = builder.build()
File "/usr/local/lib/python3.7/dist-packages/mujoco_py-2.0.2.13-py3.7.egg/mujoco_py/builder.py", line 221, in build
built_so_file_path = self._build_impl()
File "/usr/local/lib/python3.7/dist-packages/mujoco_py-2.0.2.13-py3.7.egg/mujoco_py/builder.py", line 291, in _build_impl
so_file_path = super()._build_impl()
File "/usr/local/lib/python3.7/dist-packages/mujoco_py-2.0.2.13-py3.7.egg/mujoco_py/builder.py", line 244, in _build_impl
dist.run_commands()
File "/usr/lib/python3.7/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/usr/local/lib/python3.7/dist-packages/Cython/Distutils/old_build_ext.py", line 186, in run
_build_ext.build_ext.run(self)
File "/usr/lib/python3.7/distutils/command/build_ext.py", line 340, in run
self.build_extensions()
File "/usr/local/lib/python3.7/dist-packages/mujoco_py-2.0.2.13-py3.7.egg/mujoco_py/builder.py", line 144, in build_extensions
build_ext.build_extensions(self)
File "/usr/local/lib/python3.7/dist-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
_build_ext.build_ext.build_extensions(self)
File "/usr/lib/python3.7/distutils/command/build_ext.py", line 449, in build_extensions
self._build_extensions_serial()
File "/usr/lib/python3.7/distutils/command/build_ext.py", line 474, in _build_extensions_serial
self.build_extension(ext)
File "/usr/lib/python3.7/distutils/command/build_ext.py", line 534, in build_extension
depends=ext.depends)
File "/usr/lib/python3.7/distutils/ccompiler.py", line 566, in compile
depends, extra_postargs)
File "/usr/lib/python3.7/distutils/ccompiler.py", line 348, in _setup_compile
self.mkpath(os.path.dirname(obj))
File "/usr/lib/python3.7/distutils/ccompiler.py", line 916, in mkpath
mkpath(name, mode, dry_run=self.dry_run)
File "/usr/lib/python3.7/distutils/dir_util.py", line 74, in mkpath
"could not create '%s': %s" % (head, exc.args[-1]))
distutils.errors.DistutilsFileError: could not create '/usr/local/lib/python3.7/dist-packages/mujoco_py-2.0.2.13-py3.7.egg/mujoco_py/generated/_pyxbld_2.0.2.13_37_linuxgpuextensionbuilder': Read-only file system
And here's how I'm building
It is now possible to write [temporary] files in containers via --overlay in a Singularity container:
https://sylabs.io/guides/3.7/user-guide/persistent_overlays.html