pythongribcfgrib

Load GRIB variables with "unknown" typeOfLevel using cfgrib

I'm trying to load GRIB2 files from DWD's ICON model using xarray and cfgrib. Most variables work fine, but for some (like for example CLCL), xarray.open_dataset raises the following error:

Traceback (most recent call last):
  File "/shared/conda/envs/devtools/lib/python3.8/site-packages/xarray/core/dataset.py", line 1398, in _construct_dataarray
    variable = self._variables[name]
KeyError: 'CLCL'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "DAT-2634/grib_file_loaders.py", line 276, in <module>
    df_cfgrib_sf = read_cfgrib_sf(grib_fns, nodes, params)
  File "DAT-2634/grib_file_loaders.py", line 115, in read_cfgrib_sf
    x = xr.open_dataset(
  File "/shared/conda/envs/devtools/lib/python3.8/site-packages/xarray/core/dataset.py", line 1502, in __getitem__
    return self._construct_dataarray(key)
  File "/shared/conda/envs/devtools/lib/python3.8/site-packages/xarray/core/dataset.py", line 1400, in _construct_dataarray
    _, name, variable = _get_virtual_variable(
  File "/shared/conda/envs/devtools/lib/python3.8/site-packages/xarray/core/dataset.py", line 173, in _get_virtual_variable
    ref_var = variables[ref_name]
KeyError: 'CLCL'

when I inspect the file via grib_ls I get

$ grib_ls ICON_europe_reg_0.125x0.125_2021101900_f000.grib2 | grep CLCL
2            edzw         20211019     fc           regular_ll   0            unknown      800          CLCL         grid_simple

I've read in the cfgrib issues #195 and #213 that this is probably due to the use of local codes and that I might be able to fix the issue using the appropriate code tables. DWD does provide GRIB tables for ecCodes, but I'm not sure how to use them in combination with cfgrib.

How can I use the custom code tables with cfgrib? Or is there another way to load these variables?

Solution

  • After not getting any further I've posted the same question as an issue in the cfgrib GitHub project, and got a response there: the solution is to add the path of the custom code tables to the ECCODES_DEFINITION_PATH environment variable:

    import os
from pathlib import Path
import xarray

ECCODES_DEFINITION_PATHS = [
    # The custom codes I want to use
    Path('path/to/the/custom/codes'),

    # The codes from my system's ecCodes installation
    Path('/usr/share/eccodes/definitions'),
]
os.environ['ECCODES_DEFINITION_PATH'] = ':'.join(
    str(p.resolve())
    for p in ECCODES_DEFINITION_PATHS
)

x = xarray.open_dataset(...)

    There is detailed documentation available in the ECMWF wiki.