I'm trying to load GRIB2 files from DWD's ICON model using xarray
and cfgrib
. Most variables work fine, but for some (like for example CLCL
), xarray.open_dataset
raises the following error:
Traceback (most recent call last):
File "/shared/conda/envs/devtools/lib/python3.8/site-packages/xarray/core/dataset.py", line 1398, in _construct_dataarray
variable = self._variables[name]
KeyError: 'CLCL'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "DAT-2634/grib_file_loaders.py", line 276, in <module>
df_cfgrib_sf = read_cfgrib_sf(grib_fns, nodes, params)
File "DAT-2634/grib_file_loaders.py", line 115, in read_cfgrib_sf
x = xr.open_dataset(
File "/shared/conda/envs/devtools/lib/python3.8/site-packages/xarray/core/dataset.py", line 1502, in __getitem__
return self._construct_dataarray(key)
File "/shared/conda/envs/devtools/lib/python3.8/site-packages/xarray/core/dataset.py", line 1400, in _construct_dataarray
_, name, variable = _get_virtual_variable(
File "/shared/conda/envs/devtools/lib/python3.8/site-packages/xarray/core/dataset.py", line 173, in _get_virtual_variable
ref_var = variables[ref_name]
KeyError: 'CLCL'
when I inspect the file via grib_ls
I get
$ grib_ls ICON_europe_reg_0.125x0.125_2021101900_f000.grib2 | grep CLCL
2 edzw 20211019 fc regular_ll 0 unknown 800 CLCL grid_simple
I've read in the cfgrib
issues #195 and #213 that this is probably due to the use of local codes and that I might be able to fix the issue using the appropriate code tables. DWD does provide GRIB tables for ecCodes, but I'm not sure how to use them in combination with cfgrib
.
How can I use the custom code tables with cfgrib
? Or is there another way to load these variables?
After not getting any further I've posted the same question as an issue in the cfgrib
GitHub project, and got a response there: the solution is to add the path of the custom code tables to the ECCODES_DEFINITION_PATH
environment variable:
import os
from pathlib import Path
import xarray
ECCODES_DEFINITION_PATHS = [
# The custom codes I want to use
Path('path/to/the/custom/codes'),
# The codes from my system's ecCodes installation
Path('/usr/share/eccodes/definitions'),
]
os.environ['ECCODES_DEFINITION_PATH'] = ':'.join(
str(p.resolve())
for p in ECCODES_DEFINITION_PATHS
)
x = xarray.open_dataset(...)
There is detailed documentation available in the ECMWF wiki.