How to join data from multiple netCDF files with xarray in Python?

Question

I'm trying to open multiple netCDF files with xarray in Python. The files have data with same shape and I want to join them, creating a new dimension.

I tried to use concat_dim argument for xarray.open_mfdataset(), but it doesn't work as expected. An example is given below, which open two files with temperature data for 124 times, 241 latitudes and 480 longitudes:

DS = xr.open_mfdataset( 'eraINTERIM_t2m_*.nc', concat_dim='cases' )
da_t2m = DS.t2m

print( da_t2m )

With this code, I expect that the result data array will have a shape like (cases: 2, time: 124, latitude: 241, longitude: 480). However, its shape was (cases: 2, time: 248, latitude: 241, longitude: 480). It creates a new dimension, but also sums the leftmost dimension: 'time' dimension of two datasets. I was wondering whether it's an error from 'xarray.open_mfdateset' or it's an expected behavior because 'time' dimension is UNLIMITED for both datasets.

Is there a way to join data from these files directly using xarray and get the above expected return?

Thank you.

Mateus

Answer 1

Extending from my comment I would try this:

def preproc(ds):
    ds = ds.assign({'stime': (['time'], ds.time)}).drop('time').rename({'time': 'ntime'})
    # we might need to tweak this a bit further, depending on the actual data layout
    return ds

DS = xr.open_mfdataset( 'eraINTERIM_t2m_*.nc', concat_dim='cases', preprocess=preproc)

The good thing here is, that you keep the original time coordinate in stime while renaming the original dimension (time -> ntime).

If everything works well, you should get resulting dimensions as (cases, ntime, latitude, longitude).

Disclaimer: I do similar in a loop with a final concat (wich works very well), but did not test the preprocess-approach.