javadatasetnetcdfscientific-computingncml

How to create a netCDF-4 file with a variable length variable from ncml?

I'm trying to write a netCDF-4 file with a variable length variable.

Ideally I'd have used the ncgen utility, but it does not support ncml input. Therefore, I'm using the netCDF-java lib 4.6.6. The following ncml without a variable length variable produces a workable netCDF-4 file:

java -Xmx1g -classpath ~/dump/netcdfAll-4.6.6.jar ucar.nc2.dataset.NetcdfDataset -in test.ncml -out test.nc4 -netcdf4

This is the corresponding ncml/xml:

<?xml version="1.0" encoding="UTF-8"?>
<netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://what.no">
  <group name="data">
    <dimension name="number_packets" length="6"/>
    <variable name="packet_time_utc" shape="number_packets" type="double">
    </variable>
  </group>
</netcdf>

If I'm adding another unlimited dimension to the ncml, failure occurs

<?xml version="1.0" encoding="UTF-8"?>
<netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://what.no">
  <group name="data">
    <dimension name="number_packets" length="6" isVariableLength="true" isShared="false"/>
    <variable name="packet_time_utc" shape="number_packets" type="double">
    </variable>
  </group>
</netcdf>

According to the ncml schema, the definition of the dimension is correct. Error msg NetCDF: Variable not found:

NetcdfDatataset read from test-vlen.ncml write to test-Vlen.nc4 java.io.IOException: -49: NetCDF: Variable not found
        at ucar.nc2.jni.netcdf.Nc4Iosp.writeData(Nc4Iosp.java:2835)
        at ucar.nc2.jni.netcdf.Nc4Iosp.writeData(Nc4Iosp.java:2789)
        at ucar.nc2.NetcdfFileWriter.write(NetcdfFileWriter.java:954)
        at ucar.nc2.NetcdfFileWriter.write(NetcdfFileWriter.java:938)
        at ucar.nc2.FileWriter2.copyAll(FileWriter2.java:431)
        at ucar.nc2.FileWriter2.copyVarData(FileWriter2.java:384)
        at ucar.nc2.FileWriter2.write(FileWriter2.java:199)
        at ucar.nc2.dataset.NetcdfDataset.main(NetcdfDataset.java:1888)
Exception in thread "main" java.io.IOException: -101: NetCDF: HDF error
        at ucar.nc2.jni.netcdf.Nc4Iosp.close(Nc4Iosp.java:289)
        at ucar.nc2.NetcdfFileWriter.abort(NetcdfFileWriter.java:1032)
        at ucar.nc2.FileWriter2.write(FileWriter2.java:207)
        at ucar.nc2.dataset.NetcdfDataset.main(NetcdfDataset.java:1888)

Specifying length="*", as recommended here, results in a different NumberFormatException error.

I'm using the latest versions available:

The error is about the failure to find the variable (FileWrite wants to copy all variables). However, I'm stuck trying to find the origin of the error.

Solution

  • I reproduced this with the netCDF-java lib branch 5.0.0; commit 0cc266d, but the error is a little clearer:

    With your second example: 

    java.lang.IllegalArgumentException: Dimensions added to a group must be shared.

    Upon removing isShared="false"

    java.lang.IllegalArgumentException: variable length dimension cannot be shared or unlimited

    It looks to me that there is no support just yet.