I'd like to write software that attaches extra metadata to a .zst file by appending an empty file that stores information in metadata. If it was gzip, I could use "comment" section in header, but I couldn't find any hint in a documentation that such a field is present in .zst format. What else could I "abuse" for that purpose? Would dictionary data be a good fit? If yes, how can I craft an empty .zstd file that stores metadata in a compression dictionary?
A Zstandard file is made of one or more frames. Typical files include a single frame of type Zstandard frame, but it is not mandatory.
Moreover, there exist one frame named skippable frame that can be used for user-defined metadata. This seems to be exactly what you are looking for.
For example, assuming you already have a valid Zstandard file named file.zst, the following bash code would append a `skippable frame* to the file:
$ echo -ne '\x50\x2a\x4d\x18' >> file.zst # Magic_Number
$ echo -ne '\x10\x00\x00\x00' >> file.zst # Frame_Size (length of User_Data)
$ echo -ne 'Your custom data' >> file.zst # User_Data
You can verify that the resulting file is valid with zstd -l which should report one skippable frame:
$ zstd -l file.zst
Frames Skips Compressed Uncompressed Ratio Check Filename
2 1 41 B XXH64 file.zst