xmlnetcdfthreddsncml

NcML aggregation of remote 3D and 4D variables?

Can NcML be used to aggregate 3D and 4D grids? I wasn't sure since they differ in the number of dimensions. e.g. sea surface height (ssh) and water temperature, where ssh has three dimensions [time, lat, lon] and temp has four dimensions [time, depth, lat, lon]? My tests have been unsuccessful so my hunch is that I must break the 3D and 4D variables into separate catalogs. But I'm hoping others might have alternative suggestions?

While I tried using a "union" aggregation with the snippet below, the time dimension did not get mapped appropriately because the 3D variables begin 2008-12-28 and the 4D variables begin 2008-05-08:

<?xml version="1.0" encoding="UTF-8"?>
<netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2">
  <attribute name="title" value="HYCOM test aggregation #1"/>
  <aggregation type="union">
    <!-- These are the 3D variables: -->
    <netcdf location="dods://apdrc.soest.hawaii.edu:80/dods/public_data/Model_output/HYCOM/global/hycom2d"/>
    <!-- These are the 4D variables: -->
    <netcdf location="dods://apdrc.soest.hawaii.edu:80/dods/public_data/Model_output/HYCOM/global/hycomT"/>
    <netcdf location="dods://apdrc.soest.hawaii.edu:80/dods/public_data/Model_output/HYCOM/global/hycomS"/>
    <netcdf location="dods://apdrc.soest.hawaii.edu:80/dods/public_data/Model_output/HYCOM/global/hycomU"/>
    <netcdf location="dods://apdrc.soest.hawaii.edu:80/dods/public_data/Model_output/HYCOM/global/hycomV"/> 
  </aggregation>
</netcdf>

I then tried a "joinExisting" aggregation on the "time" dimension, but this only works if the datasets each contain the same variables (which they don't). Depending on which dataset I list first in my aggregation, either the 3D or 4D variables get excluded in the following example:

<?xml version="1.0" encoding="UTF-8"?>
<netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2">
  <attribute name="title" value="HYCOM test aggregation #2"/>
  <aggregation dimName="time" type="joinExisting">
    <!-- These are the 3D variables: -->
    <netcdf location="dods://apdrc.soest.hawaii.edu:80/dods/public_data/Model_output/HYCOM/global/hycom2d"/>
    <!-- These are the 4D variables: -->
    <aggregation type="union">
      <netcdf location="dods://apdrc.soest.hawaii.edu:80/dods/public_data/Model_output/HYCOM/global/hycomT"/>
      <netcdf location="dods://apdrc.soest.hawaii.edu:80/dods/public_data/Model_output/HYCOM/global/hycomS"/>
      <netcdf location="dods://apdrc.soest.hawaii.edu:80/dods/public_data/Model_output/HYCOM/global/hycomU"/>
      <netcdf location="dods://apdrc.soest.hawaii.edu:80/dods/public_data/Model_output/HYCOM/global/hycomV"/> 
    </aggregation>
  </aggregation>
</netcdf>

So, is there no way to aggregate these datasets? Must I keep the 3D and 4D variables separate?

Thanks!, John Maurer Pacific Islands Ocean Observing System (PacIOOS) University of Hawaii at Manoa

Solution

  • John,

    Since the files you want to join have different time coordinates but have the same name, you need to rename one of them. I would have thought that this simple NcML would have worked, simply renaming the dimension and name of the time dimension and time variable in the 3D data

    <?xml version="1.0" encoding="UTF-8"?>
<netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2">
    <attribute name="title" value="HYCOM test aggregation #1"/>
    <aggregation type="union">
        <!-- These are the 3D variables: -->
        <netcdf location="dods://apdrc.soest.hawaii.edu:80/dods/public_data/Model_output/HYCOM/global/hycom2d">
            <dimension name="time2d" orgName="time"/>
            <variable name="time2d" orgName="time"/>
        </netcdf>
        <!-- These are the 4D variables: -->
        <netcdf location="dods://apdrc.soest.hawaii.edu:80/dods/public_data/Model_output/HYCOM/global/hycomT"/>
        <netcdf location="dods://apdrc.soest.hawaii.edu:80/dods/public_data/Model_output/HYCOM/global/hycomS"/>
        <netcdf location="dods://apdrc.soest.hawaii.edu:80/dods/public_data/Model_output/HYCOM/global/hycomU"/>
        <netcdf location="dods://apdrc.soest.hawaii.edu:80/dods/public_data/Model_output/HYCOM/global/hycomV"/> 
    </aggregation>
</netcdf>

    but it does not, because somehow NetCDF-Java seems to be adding an attribute _CoordinateAxes with the values "time lon lat" before we change the variable and dimension names. So if we remove that attribute from the 3D data, it works:

    <?xml version="1.0" encoding="UTF-8"?>
<netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2">
    <attribute name="title" value="HYCOM test aggregation #1"/>
    <aggregation type="union">
        <!-- These are the 3D variables: -->
        <netcdf location="dods://apdrc.soest.hawaii.edu:80/dods/public_data/Model_output/HYCOM/global/hycom2d">
            <dimension name="time2d" orgName="time"/>
            <variable name="time2d" orgName="time"/>
            <variable name="qtot">
                <remove type="attribute" name="_CoordinateAxes"/>
            </variable>
            <variable name="emp">
                <remove type="attribute" name="_CoordinateAxes"/>
            </variable>
            <variable name="t_trend">
                <remove type="attribute" name="_CoordinateAxes"/>
            </variable>
            <variable name="s_trend">
                <remove type="attribute" name="_CoordinateAxes"/>
            </variable>
            <variable name="ssh">
                <remove type="attribute" name="_CoordinateAxes"/>
            </variable>
            <variable name="mld">
                 <remove type="attribute" name="_CoordinateAxes"/>
            </variable>
            <variable name="mlp">
                 <remove type="attribute" name="_CoordinateAxes"/>
            </variable>
        </netcdf>
        <!-- These are the 4D variables: -->
        <netcdf location="dods://apdrc.soest.hawaii.edu:80/dods/public_data/Model_output/HYCOM/global/hycomT"/>
        <netcdf location="dods://apdrc.soest.hawaii.edu:80/dods/public_data/Model_output/HYCOM/global/hycomS"/>
        <netcdf location="dods://apdrc.soest.hawaii.edu:80/dods/public_data/Model_output/HYCOM/global/hycomU"/>
        <netcdf location="dods://apdrc.soest.hawaii.edu:80/dods/public_data/Model_output/HYCOM/global/hycomV"/> 
    </aggregation>
</netcdf>

    Here's a a screenshot from ToolsUI of the resulting dataset, where you can see the 3D and 4D variables: