pythonnetcdfpython-xarrayopendap

How to open multiple NetCDF files with xarray via OpenDAP?

I'm trying to open multiple NetCDF files hosted on OpenDAP on a THREDDS server, using xarray.open_mfdataset(), but I get an error. If I open only one file (but still with open_mfdataset()), it works, if I open two, it doesn't.

For example, this works just fine:

import xarray as xr
chunks = {'time' : 1, 'depth' : 1}
paths  = [
    'http://thredds.met.no/thredds/dodsC/fou-hi/norkyst800m-1h/NorKyst-800m_ZDEPTHS_his.an.2017022000.nc',
    ]
d = xr.open_mfdataset(paths, chunks = chunks)

while this doesn't:

import xarray as xr
chunks = {'time' : 1, 'depth' : 1}
paths  = [
    'http://thredds.met.no/thredds/dodsC/fou-hi/norkyst800m-1h/NorKyst-800m_ZDEPTHS_his.an.2017022000.nc',
    'http://thredds.met.no/thredds/dodsC/fou-hi/norkyst800m-1h/NorKyst-800m_ZDEPTHS_his.an.2017022100.nc',
    ]
d = xr.open_mfdataset(paths, chunks = chunks)

I'm running this in a jupyter notebook, and I don't get any output in the notebook, it just keeps running, while in the terminal it prints the following 

CURL Error: Failed initialization
curl error details:
CURL Error: Failed initialization
CURL Error: Failed initialization
CURL Error: Failed initialization
syntax error, unexpected WORD_WORD, expecting SCAN_ATTR or SCAN_DATASET or SCAN_ERROR
context: !�zF^@L������@L�Ы�J�@L�y@L�G`Lp@L�X�ڭ@L�/��a@L���@L��{�:@@L�Hl�D@L���i@L�f���@L�>����@L�x��f@L��DA�h@L�����@L��ڭ�M@L�u���@M
CURL Error: Failed initialization
curl error details:
CURL Error: Failed initialization
CURL Error: Failed initialization
syntax error, unexpected WORD_WORD, expecting $end
context: Dataset { Structure { Float64 lon[Y = 902][X = 2602]; } lon;} fou-hi/norkyst800m-1h/NorKyst-800m_ZDEPTHS_his.an.2017022100.nc;Data^:
CURL Error: Failed initialization
CURL Error: Failed initialization
CURL Error: Failed initialization
syntax error, unexpected WORD_WORD, expecting SCAN_ATTR or SCAN_DATASET or SCAN_ERROR
context: �@��&��D^@�h�N�{@��I$�@B�

and from there on, it just degenerates into printing more garbage.

Isn't this supposed to work just fine?

Edit:

I wasn't aware of this before, but it turns out the netCDF4 library also supports opening multiple paths as one dataset. I'm not sure if this is relevant, as I don't know if xarray and netCDF4 use the same backends, but in any case, the following works just fine. At least that suggests the problem isn't at the server side.

import netCDF4 as nc
d = nc.MFDataset([
    'http://thredds.met.no/thredds/dodsC/fou-hi/norkyst800m-1h/NorKyst-800m_ZDEPTHS_his.an.2017022000.nc',
    'http://thredds.met.no/thredds/dodsC/fou-hi/norkyst800m-1h/NorKyst-800m_ZDEPTHS_his.an.2017022100.nc',
])
Solution

  • This looks like a bug of some sort to me -- you are using the API properly and there is no inherent reason why this shouldn't work. But I don't know which system is at fault -- most likely it's either the netCDF-C OpenDAP reader or the remote OpenDAP server.