Edit: Raw files for the years 1996 and 2005 for testing can be found here.
I have output from two modelling scenarios in a climate risk model (evergreen and control). I want to find differences between the modelling scenarios in extreme (min and max) and average values for each year to plot this on maps with Python Cartopy. I have used both CDO and NCO to get these NetCDF files, to see if one of these two programs results in the side location conflict or if it's a problem with the data itself.
I take three steps:
Somehow this leads to a side location conflict in both CDO and NCO in the maximum differences file, but not in the minimum or average. Since the minimum, average and maximum files have the same input files, what could be causing this difference and side location conflict in only one of the output files?
Below the NCO code. The ... indicate lines that I left out here for readability, but that cover the years 1997-2004:
echo Creating yearly average of AFDR Evergreen_Evergreen data
ncwa -a Times AFDR_WRF/Evergreen/evergreen_evergreen_AFDR_WRF_Scandinavia_1996.nc AFDR_WRF/Evergreen/Average/aver_evergreen_evergreen_AFDR_WRF_Scandinavia_1996.nc
...
ncwa -a Times AFDR_WRF/Evergreen/evergreen_evergreen_AFDR_WRF_Scandinavia_2005.nc AFDR_WRF/Evergreen/Average/aver_evergreen_evergreen_AFDR_WRF_Scandinavia_2005.nc
echo Concatenating yearly average of Evergreen data files
ncecat AFDR_WRF/Evergreen/Average/aver_evergreen_evergreen*.nc -O AFDR_WRF/Evergreen/Average/average_evergreen_evergreen_AFDR_1996-2005.nc
echo Creating yearly maximum of AFDR Evergreen_Evergreen data
ncra -y max AFDR_WRF/Evergreen/evergreen_evergreen_AFDR_WRF_Scandinavia_1996.nc AFDR_WRF/Evergreen/Maximum/max_evergreen_evergreen_AFDR_WRF_Scandinavia_1996.nc
...
ncra -y max AFDR_WRF/Evergreen/evergreen_evergreen_AFDR_WRF_Scandinavia_2005.nc AFDR_WRF/Evergreen/Maximum/max_evergreen_evergreen_AFDR_WRF_Scandinavia_2005.nc
echo Concatenating yearly maximum of Evergreen data files
ncecat AFDR_WRF/Evergreen/Maximum/max_evergreen_evergreen*.nc -O AFDR_WRF/Evergreen/Maximum/maximum_evergreen_evergreen_AFDR_1996-2005.nc
echo Creating yearly minimum of AFDR Evergreen_Evergreen data
ncra -y min AFDR_WRF/Evergreen/evergreen_evergreen_AFDR_WRF_Scandinavia_1996.nc AFDR_WRF/Evergreen/Minimum/min_evergreen_evergreen_AFDR_WRF_Scandinavia_1996.nc
...
ncra -y min AFDR_WRF/Evergreen/evergreen_evergreen_AFDR_WRF_Scandinavia_2005.nc AFDR_WRF/Evergreen/Minimum/min_evergreen_evergreen_AFDR_WRF_Scandinavia_2005.nc
echo Concatenating yearly minimum of Evergreen data files
ncecat AFDR_WRF/Evergreen/Minimum/min_evergreen_evergreen*.nc -O AFDR_WRF/Evergreen/Minimum/minimum_evergreen_evergreen_AFDR_1996-2005.nc
echo Creating yearly average of AFDR Control_Control data
ncwa -a Times AFDR_WRF/Control/control_control_AFDR_WRF_Scandinavia_1996.nc AFDR_WRF/Control/Average/aver_control_control_AFDR_WRF_Scandinavia_1996.nc
...
ncwa -a Times AFDR_WRF/Control/control_control_AFDR_WRF_Scandinavia_2005.nc AFDR_WRF/Control/Average/aver_control_control_AFDR_WRF_Scandinavia_2005.nc
echo Concatenating yearly average of Control data files
ncecat AFDR_WRF/Control/Average/aver_control_control*.nc -O AFDR_WRF/Control/Average/average_control_control_AFDR_1996-2005.nc
echo Creating yearly maximum of AFDR Control_Control data
ncra -y max AFDR_WRF/Control/control_control_AFDR_WRF_Scandinavia_1996.nc AFDR_WRF/Control/Maximum/max_control_control_AFDR_WRF_Scandinavia_1996.nc
...
ncra -y max AFDR_WRF/Control/control_control_AFDR_WRF_Scandinavia_2005.nc AFDR_WRF/Control/Maximum/max_control_control_AFDR_WRF_Scandinavia_2005.nc
echo Concatenating yearly maximum of Control data files
ncecat AFDR_WRF/Control/Maximum/max_control_control*.nc -O AFDR_WRF/Control/Maximum/maximum_control_control_AFDR_1996-2005.nc
echo Creating yearly minimum of AFDR Control_Control data
ncra -y min AFDR_WRF/Control/control_control_AFDR_WRF_Scandinavia_1996.nc AFDR_WRF/Control/Minimum/min_control_control_AFDR_WRF_Scandinavia_1996.nc
...
ncra -y min AFDR_WRF/Control/control_control_AFDR_WRF_Scandinavia_2005.nc AFDR_WRF/Control/Minimum/min_control_control_AFDR_WRF_Scandinavia_2005.nc
echo Concatenating yearly minimum of Control data files
ncecat AFDR_WRF/Control/Minimum/min_control_control*.nc -O AFDR_WRF/Control/Minimum/minimum_control_control_AFDR_1996-2005.nc
echo Differences and timeaverages for min, max and average in AFDR
ncdiff AFDR_WRF/Evergreen/Minimum/minimum_evergreen_evergreen_AFDR_1996-2005.nc AFDR_WRF/Control/Minimum/minimum_control_control_AFDR_1996-2005.nc AFDR_WRF/DifferenceMin/yearmin_evergreen_evergreen-control_control_AFDR_1996-2005.nc
ncdiff AFDR_WRF/Evergreen/Average/average_evergreen_evergreen_AFDR_1996-2005.nc AFDR_WRF/Control/Average/average_control_control_AFDR_1996-2005.nc AFDR_WRF/DifferenceAvg/yearavg_evergreen_evergreen-control_control_AFDR_1996-2005.nc
ncdiff AFDR_WRF/Evergreen/Maximum/maximum_evergreen_evergreen_AFDR_1996-2005.nc AFDR_WRF/Control/Maximum/maximum_control_control_AFDR_1996-2005.nc AFDR_WRF/DifferenceMax/yearmax_evergreen_evergreen-control_control_AFDR_1996-2005.nc
ncwa -a record AFDR_WRF/DifferenceMin/yearmin_evergreen_evergreen-control_control_AFDR_1996-2005.nc AFDR_WRF/timavg_yearmin_evergreen_evergreen-control_control_AFDR_1996-2005.nc
ncwa -a record AFDR_WRF/DifferenceAvg/yearavg_evergreen_evergreen-control_control_AFDR_1996-2005.nc AFDR_WRF/timavg_yearavg_evergreen_evergreen-control_control_AFDR_1996-2005.nc
ncwa -a record AFDR_WRF/DifferenceMax/yearmax_evergreen_evergreen-control_control_AFDR_1996-2005.nc AFDR_WRF/timavg_yearmax_evergreen_evergreen-control_control_AFDR_1996-2005.nc
Below the same steps written in CDO:
cdo -O mergetime AFDR_WRF/Control/control_control_AFDR*.nc AFDR_WRF/Control/control_control_AFDR_1996-2005.nc
cdo -O -yearmin AFDR_WRF/Control/control_control_AFDR_1996-2005.nc AFDR_WRF/YearMin/yearmin_control_control_AFDR_1996-2005.nc
cdo -O -yearavg AFDR_WRF/Control/control_control_AFDR_1996-2005.nc AFDR_WRF/YearAvg/yearavg_control_control_AFDR_1996-2005.nc
cdo -O -yearmax AFDR_WRF/Control/control_control_AFDR_1996-2005.nc AFDR_WRF/YearMax/yearmax_control_control_AFDR_1996-2005.nc
cdo -O mergetime AFDR_WRF/Evergreen/evergreen_evergreen_AFDR*.nc AFDR_WRF/Evergreen/evergreen_evergreen_AFDR_1996-2005.nc
cdo -O -yearmin AFDR_WRF/Evergreen/evergreen_evergreen_AFDR_1996-2005.nc AFDR_WRF/YearMin/yearmin_evergreen_evergreen_AFDR_1996-2005.nc
cdo -O -yearavg AFDR_WRF/Evergreen/evergreen_evergreen_AFDR_1996-2005.nc AFDR_WRF/YearAvg/yearavg_evergreen_evergreen_AFDR_1996-2005.nc
cdo -O -yearmax AFDR_WRF/Evergreen/evergreen_evergreen_AFDR_1996-2005.nc AFDR_WRF/YearMax/yearmax_evergreen_evergreen_AFDR_1996-2005.nc
cdo -O -sub AFDR_WRF/YearMin/yearmin_evergreen_evergreen_AFDR_1996-2005.nc AFDR_WRF/YearMin/yearmin_control_control_AFDR_1996-2005.nc AFDR_WRF/DifferenceMin/yearmin_evergreen_evergreen-control_control_AFDR_1996-2005.nc
cdo -O -sub AFDR_WRF/YearAvg/yearavg_evergreen_evergreen_AFDR_1996-2005.nc AFDR_WRF/YearAvg/yearavg_control_control_AFDR_1996-2005.nc AFDR_WRF/DifferenceAvg/yearavg_evergreen_evergreen-control_control_AFDR_1996-2005.nc
cdo -O -sub AFDR_WRF/YearMax/yearmax_evergreen_evergreen_AFDR_1996-2005.nc AFDR_WRF/YearMax/yearmax_control_control_AFDR_1996-2005.nc AFDR_WRF/DifferenceMax/yearmax_evergreen_evergreen-control_control_AFDR_1996-2005.nc
cdo -O -timavg AFDR_WRF/DifferenceMin/yearmin_evergreen_evergreen-control_control_AFDR_1996-2005.nc AFDR_WRF/timavg_yearmin_evergreen_evergreen-control_control_AFDR_1996-2005.nc
cdo -O -timavg AFDR_WRF/DifferenceAvg/yearavg_evergreen_evergreen-control_control_AFDR_1996-2005.nc AFDR_WRF/timavg_yearavg_evergreen_evergreen-control_control_AFDR_1996-2005.nc
cdo -O -timavg AFDR_WRF/DifferenceMax/yearmax_evergreen_evergreen-control_control_AFDR_1996-2005.nc AFDR_WRF/timavg_yearmax_evergreen_evergreen-control_control_AFDR_1996-2005.nc
Last the Python code:
from netCDF4 import Dataset
import cartopy.crs as ccrs
import numpy as np
import matplotlib.pyplot as plt
import geopandas
dataset_afdr_evergreen_evergreen_control_control_min = Dataset("path/to/data","r")
dataset_afdr_evergreen_evergreen_control_control_avg = Dataset("path/to/data","r")
dataset_afdr_evergreen_evergreen_control_control_max = Dataset("path/to/data","r")
data_afdr_evergreen_evergreen_control_control_min = (
dataset_afdr_evergreen_evergreen_control_control_min["fire_behaviour_index"][0, :, :])
data_afdr_evergreen_evergreen_control_control_avg = (
dataset_afdr_evergreen_evergreen_control_control_avg["fire_behaviour_index"][0, :, :])
data_afdr_evergreen_evergreen_control_control_max = (
dataset_afdr_evergreen_evergreen_control_control_max["fire_behaviour_index"][0, :, :])
lons_afdr = dataset_afdr_evergreen_evergreen_control_control_min["XLAT"][:, :]
lons_afdr = dataset_afdr_evergreen_evergreen_control_control_min["XLONG"][:, :]
min_evergreen_evergreen_control_control = -50
max_evergreen_evergreen_control_control = 50
levels_evergreen_evergreen_control_control = np.linspace(
min_evergreen_evergreen_control_control,
max_evergreen_evergreen_control_control,
max_evergreen_evergreen_control_control + 1,
)
# Create a plot
fig, axs = plt.subplots(
nrows=1,
ncols=3,
subplot_kw={"projection": ccrs.Mercator()},
figsize=(90, 70),
layout="constrained",
# axis="off",
)
extent = [4.5, 31.9, 54.7, 71.24]
afdr_evergreen_evergreen_control_control_min = axs[0].contourf(
lons_afdr,
lats_afdr,
data_afdr_evergreen_evergreen_control_control_min,
60,
transform=ccrs.PlateCarree(),
levels=levels_evergreen_evergreen_control_control,
cmap="coolwarm",
extend="both",
)
axs[0].set_extent(extent)
axs[0].axis("off")
axs[0].set_title(
"AFDR evergreen-control landuse+climate mean yearly minimum",
loc="left",
fontsize=65,
)
cbar1 = fig.colorbar(
afdr_evergreen_evergreen_control_control_min,
ax=axs[0],
ticks=[-50, -40, -30, -20, -10, 0, 10, 20, 30, 40, 50],
orientation="vertical",
shrink=1,
pad=0.01,
)
cbar1.set_label('Average difference in yearly minimum risk index', size=65, labelpad=15)
cbar1.ax.tick_params(labelsize=60)
afdr_evergreen_evergreen_control_control_avg = axs[1].contourf(
lons_afdr,
lats_afdr,
data_afdr_evergreen_evergreen_control_control_avg,
60,
transform=ccrs.PlateCarree(),
levels=levels_evergreen_evergreen_control_control,
cmap="coolwarm",
extend="both",
)
axs[1].set_extent(extent)
axs[1].axis("off")
axs[1].set_title(
"AFDR evergreen-control landuse+climate mean yearly average",
loc="left",
fontsize=65,
)
cbar2 = fig.colorbar(
afdr_evergreen_evergreen_control_control_avg,
ax=axs[1],
ticks=[-50, -40, -30, -20, -10, 0, 10, 20, 30, 40, 50],
orientation="vertical",
shrink=1,
pad=0.01,
)
cbar2.set_label('Average difference in yearly average risk index', size=65, labelpad=15)
cbar2.ax.tick_params(labelsize=60)
afdr_evergreen_evergreen_control_control_max = axs[2].contourf(
lons_afdr,
lats_afdr,
data_afdr_evergreen_evergreen_control_control_max,
60,
transform=ccrs.PlateCarree(),
levels=levels_evergreen_evergreen_control_control,
cmap="coolwarm",
extend="both",
)
axs[2].set_extent(extent)
axs[2].axis("off")
axs[2].set_title(
"AFDR evergreen-control landuse+climate mean yearly maximum",
loc="left",
fontsize=65,
)
cbar3 = fig.colorbar(
afdr_evergreen_evergreen_control_control_max,
ax=axs[2],
ticks=[-50, -40, -30, -20, -10, 0, 10, 20, 30, 40, 50],
orientation="vertical",
shrink=1,
pad=0.01,
)
cbar3.set_label('Average difference in yearly maximum risk index', size=65, labelpad=15)
cbar3.ax.tick_params(labelsize=60)
plt.savefig("/path/to/figure.png")
The issue was caused by a bug in the GEOS version used in Shapely, as pointed out by RuthC. Installing conda with the latest versions of all packages solved the problem!