I need to store a 512^3 array on disk in some way and I'm currently using HDF5. Since the array is sparse a lot of disk space gets wasted.
Does HDF5 provide any support for sparse array ?
Chunked datasets (H5D_CHUNKED) allow sparse storage but depending on your data, the overhead may be important.
Take a typical array and try both sparse and non-sparse and then compare the file sizes, then you will see if it is really worth.