posixtemporary-filestmpfs

Where should an application targeting Linux store temporary files that are expected to be larger than memory?


With Ubuntu 24.10 switching to a tmpfs-based (i.e. in-memory) /tmp, having a /tmp that is not usefully larger than memory, or even one that is smaller than memory, is about to become a lot more common.

As an application developer, I often store temporary data in temporary files provided by tmpfile(). Usually I am doing this because I don't think it would be appropriate to keep the data in memory. Sometimes I might even know, or at least strongly suspect, that my data is too large to keep in memory, and I might be trying to store it on disk instead to keep memory usage at an acceptable level.

But if many of the systems my software is going to run on now back /tmp and thus the default tmpfile() location with memory, the strategy of putting data known to be larger than memory onto disk via tmpfile()'s default storage location can't work anymore. If I have data larger than memory, I will need to tell tmpfile() where to put it.

So my question is: Where should an application targeting Linux store temporary files that are expected to be larger than memory?

Should known-large files actually go in e.g. $XDG_CACHE_HOME? Or somewhere else? Or does XDG need a new directory type added?


Solution

  • Where should an application targeting Linux store temporary files that are expected to be larger than memory?

    Unfortunately, the "correct" place to store such files must be determined by the system admin who understands your application's requirements.

    I'd suggest you default to /var/tmp/app-name (or one of /var/{cache,lib,spool}/app-name if those are more appropriate), and make the location explicitly configurable and documented. Choose your temp file strategy — (mkdtemp/mkostemp? maildir?) — and be done with it.