Reading in multiple WRF history files #6586

[tlsw231]

This is a thread migrated over from the xarray github discussions.

What I am trying to do: Read in multiple (~3000) WRF history files.

One of the ideas suggested was the following (files below is just a list of strings with all the history filenames):

import xarray as xr
dsets = [xr.open_dataset(file, chunks={}).xwrf.postprocess() for file in files]
ds = xr.concat(dsets, dim='Time')

I get the following error: arg must be a string, datetime, list, tuple, 1-d array, or Series.

The same error persists when I try it on just one file: xr.open_dataset(files[0],chunks={}).xwrf.postprocess()

If I use .xwrf.postprocess(decode_times=False) then the command goes through but that obviously defeats the purpose as I lose the time information.

What is the simplest way to read in multiple WRF output files using xarray?

[lpilz]

Hi there, I'm not sure whether this is an error on our end - to assess this I would need a bit more error output. However, in general in order to read in multiple files, I would suggest you try using xr.open_mfdataset().

[dcherian]

have you tried it with combine="nested", concat_dim="Time? In your command above it's using by_coords but can't figure out to concatenate along Time. IIRC concat_dim is only applicable to combine="nested" and we should be raising an error but maybe not one's fixed that yet

[dcherian]

You could also try a preprocess function that is just return ds.expand_dims('Time')

[lpilz]

So what I do is that I use the xr.open_mfdataset(preprocess) kwarg. My solution to this goes like this (a bit different because I have to cut overlapping files, so this might not work out of the box):

def fix_wrf_time_coord(ds):
    _times = pd.to_datetime(ds.Times.astype(str), format="%Y-%m-%d_%H:%M:%S")
    ds.Times.values = _times
    return ds

xr.open_mfdataset(files, chunks='auto', preprocess=fix_wrf_time_coord, concat_dim="Time", combine="nested").xwrf.postprocess()

[tlsw231]

Thanks, that was helpful!

[tlsw231]

Interestingly, adding the arguments data_vars='minimal', coords='minimal', compat='override' slows down the open_mfdataset() step slightly.