I have these scripts that perform a lot of expensive computations. One optimization I do is to evaluate the expensive function on a grid, cache the grid, and interpolate for calls within the grid. It is important that the files be written out to disk so that future sessions can benefit from the cache. When this script was just for my own personal use, I could put the cache wherever. Now that I want to incorporate it into a package for wider distribution, I find that I need to have some way to know where is it kosher to store cache files for a package? Do I just need to make some decisions, or are there some procedures I should follow (e.g. let the user modify the directory at install time, at run-time, etc)?
Edit: this code will also have functions based on interpolating data from other sources. So I will need a place to store those, too, but need to know where it is standard to do so and how to detect where that is on a particular install (something hard coded at install time, or can Python modules detect where they're installed at runtime?). Point being, this location needs to be persistent in a way that I understand temporary directories not to be. Therefore, this would ideally be done inside of the package's directory structure or somewhere in the user's home directory. If in the package's directories, it is acceptable if I have to do shenanigans with permissions to make a directory that the user can modify.
I have the same requirements.
It seems like the way to go is storing to ~/.cache/<app name> (Linux).
For covering different operating systems, getting similar location can be simplified with platformdirs (src, PyPI, conda-forge), e.g.:
from platformdirs import user_cache_dir
cachedir = user_cache_dir("<app name>", "<app author>")