hdf5hdf5dotnet

How to create a 3D extendable dataset with HDF5 in .NET using HDF.Pinvoke?


I am using HDF.PInvoke with C#.

I have already developed lots of code to save 2D datasets which all works fine.

I now wish to write a series of 2D datasets (of constant size width * width) into a 3D array and I am finding that I am unable to create the 3D dataset in the first place. The data is in the form of a pitch linear array and the first thing I try to do is create a width * width * 1 dataset that is extendable in the 'Z' direction. The code follows with an indicator of where the error occurs. Any help very gratefully received...

using HDF.PInvoke;
using hid_t = System.Int64;
using hsize_t = System.UInt64;

public static void Create3DDataSet(hid_t gpID, string name, ushort[] im )
{            
    const int width = 2048;
     
    System.Diagnostics.Trace.Assert(im.Length == width * width);
    const int rank = 3;
    hsize_t[] current_dims = new hsize_t[rank] { width, width, 1 };
    hsize_t[] max_Dims = new hsize_t[rank] { width, width, H5S.UNLIMITED };

    hid_t dataSpace = HDFErrChk(H5S.create(H5S.class_t.SIMPLE));//1S
    HDFErrChk(H5S.set_extent_simple(dataSpace, rank, current_dims, max_Dims));
    hid_t typeId = HDFErrChk(H5T.copy(H5T.NATIVE_USHORT));//2T

    //Next line returns an error value of -1
    hid_t datasetId = HDFErrChk(H5D.create(gpID, name, typeId, dataSpace, 0L, 0L, 0L));//3D

    GCHandle pinnedArray = GCHandle.Alloc(im, GCHandleType.Pinned);
    HDFErrChk(H5D.write(datasetId, typeId, H5S.ALL, H5S.ALL, H5P.DEFAULT, pinnedArray.AddrOfPinnedObject()));
    pinnedArray.Free();


    HDFErrChk(H5S.close(dataSpace));//1S
    HDFErrChk(H5T.close(typeId));//2T
    HDFErrChk(H5D.close(datasetId));//3D
}
    
public static long HDFErrChk(long err)
{
    if (err < 0)
    {
       throw new HDF5Exception("Negative return value from HDF");
    }
    return err;
}

Solution

  • Thanks to Phillipe for his invaluable assistance offline with this.

    The code appears to be failing because of the use of H5S.UNLIMITED when the maximum dimensions of the scan are set. It is not clear if this is a general problem or one that is specific to the use of HDF.PInvoke.

    Either way replacing H5S.UNLIMITED with an actual number (in ulong/hsize_t format) fixes the problem that has been occurring when the dataset is created.

    For the record I have also switched around the way I define the dimensions so that the 'index' for the 2D slices is the first dimension... Also please Note there is a bunch of missing code about defining chunks and setting up memory spaces that needs to occur after the dataset has been created. I have not added this as the original question was about a problem creating a dataset, not writing to it... The modified code follows:-

    using HDF.PInvoke;
    using hid_t = System.Int64;
    using hsize_t = System.UInt64;
    
    public static void Create3DDataSet(hid_t gpID, string name, hsize_t maxLen, ushort[] im )
    {            
        const hsize_t width = 2048;
         
        System.Diagnostics.Trace.Assert(im.Length == width * width);
        const int rank = 3;
        hsize_t[] current_dims = new hsize_t[rank] { 1, width, width };
        hsize_t[] max_Dims = new hsize_t[rank] { maxLen, width, width };
    
        hid_t dataSpace = HDFErrChk(H5S.create(H5S.class_t.SIMPLE));//1S
        HDFErrChk(H5S.set_extent_simple(dataSpace, rank, current_dims, max_Dims));
        hid_t typeId = HDFErrChk(H5T.copy(H5T.NATIVE_USHORT));//2T
    
        
        hid_t datasetId = HDFErrChk(H5D.create(gpID, name, typeId, dataSpace, 0L, 0L, 0L));//3D
    
       /*  Add code to write data here  */   
    
    
        HDFErrChk(H5S.close(dataSpace));//1S
        HDFErrChk(H5T.close(typeId));//2T
        HDFErrChk(H5D.close(datasetId));//3D
    }
    
    public static long HDFErrChk(long err)
    {
        if (err < 0)
        {
           throw new HDF5Exception("Negative return value from HDF");
        }
        return err;
    }