[SOLVED] Resources available locally but "No such file or directory" when packaged and deployed with Poetry

Resources available locally but "No such file or directory" when packaged and deployed with Poetry

I use paths defined as constants, e.g. TF_CONSTS = 'consts/tf_keras_param_config.json' and they work fine during development.

However, when the package is built with Poetry (i.e. poetry build --format sdist) and deployed, these references become invalid even though the JSON files are copied along with the Python scripts:

FileNotFoundError: [Errno 2] No such file or directory: 'some_package/consts/tf_keras_param_config.json'

Why?

Solution

Relative paths work fine during development, but - when the code is packaged and then installed - they must be translated to absolute ones and these must be determined dynamically (they depend on the location of the installed package):

import pkg_resources

# necessary for the path to resolve properly when package is installed
TF_CONSTS = pkg_resources.resource_filename("some_package", "consts/tf_keras_param_config.json")

Consider also importlib.resources:

This module provides functionality similar to pkg_resources Basic Resource Access without the performance overhead of that package. This makes reading resources included in packages easier, with more stable and consistent semantics.