buildtimestamparchivetar

How to create a tar file that omits timestamps for its contents?


Is there a way to create a .tar file that omits the values of atime/ctime/mtime for its files/directories?

Why do we want to do this?

We have a step in our build process that generates a directory of artifacts that gets packaged into a tarfile. We expect that build step to be idempotent -- given the same inputs, it produces exactly the same files/output each time.

Ideally, we would like also like the step to be bitwise idempotent across clean builds, so that we can use hashes of successive builds to check that nothing has changed. But because tar files include timestamps (atime/ctime/mtime) for each entry, the tar files created by that build step are never bitwise identical to the previous run, even though the contents of every file inside the archive are bitwise identical.

Is there a way to generate a tarfile that omits the timestamps of its entries, so that the step that generates the archive could be bitwise idempotent? (We want to leverage other file metadata that tar preserves, such as file mode bits and symlinks.)


Solution

  • To have a truly idempotent tar, mtime is a good step but not enough. You also need to set the sort order, the owner and group (together with their mapping) and a proper timezone for mtime (since otherwise you're gonna have issues as well between Mac and Linux).

    I ended up with

    tar --sort=name --owner=root:0 --group=root:0 --mtime='UTC 2019-01-01' ... | gzip -n