Loading a cube from netcdf with a wildcard produces MergeError

[am-thyst]

🐛 Bug Report

I have a collection of files containing the same diagnostics but different times (output from the UM). I can load all of the times into one cube using .pp files:

iris.load_cube("umnsaa_pa*.pp","surface_upward_latent_heat_flux")
<iris 'Cube' of surface_upward_latent_heat_flux / (W m-2) (time: 120; latitude: 550; longitude: 380)>

However, when importing the exact same data from netcdf, I get a MergeError:

iris.load_cube("umnsaa_pa*.nc","surface_upward_latent_heat_flux")


Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/miniconda3/envs/py310_env/lib/python3.10/site-packages/iris/cube.py", line 391, in merge_cube
    proto_cube.register(cube, error_on_mismatch=True)
  File "/home/miniconda3/envs/py310_env/lib/python3.10/site-packages/iris/_merge.py", line 1361, in register
    match = coord_payload.match_signature(
  File "/home/miniconda3/envs/py310_env/lib/python3.10/site-packages/iris/_merge.py", line 284, in match_signature
    raise iris.exceptions.MergeError(msgs)
iris.exceptions.MergeError: failed to merge into a single cube.
  Coordinates in cube.dim_coords differ: time.
  Coordinates in cube.aux_coords (non-scalar) differ: forecast_period.

I can fix this by using:

iris.load("umnsaa_pa*.nc","surface_upward_latent_heat_flux").concatenate()[0]
<iris 'Cube' of surface_upward_latent_heat_flux / (W m-2) (time: 120; latitude: 550; longitude: 380)>

How To Reproduce

Steps to reproduce the behaviour:

from iris.cube import Cube
from iris.coords import DimCoord
from cf_units import Unit
import numpy as np
import iris

times = np.array([418344,418345])

for x in [0,2]:
    T = times + x
    time = DimCoord(T, standard_name='time',
                    units=Unit('hours since 1970-01-01 00:00:00', calendar='gregorian'))
    
    latitude = DimCoord(np.linspace(-90, 90, 4),
                        standard_name='latitude',
                        units='degrees')
    longitude = DimCoord(np.linspace(45, 360, 8),
                         standard_name='longitude',
                         units='degrees')
    
    cube = Cube(np.zeros((2, 4, 8), np.float32),
                dim_coords_and_dims=[(time,0),(latitude, 1),
                                     (longitude, 2)])
    
    iris.save(cube,"cube_{0}.pp".format(x))
    iris.save(cube,"cube_{0}.nc".format(x))

iris.load_cube("cube_*.pp")
# should output a cube with all of the times
iris.load_cube("cube_*.nc")
# should output the error

Environment

• OS & Version: CentOS Linux, version 7 (core)
• Iris Version: 3.2.1

[github-actions]

In order to maintain a backlog of relevant issues, we automatically label them as stale after 500 days of inactivity.

If this issue is still important to you, then please comment on this issue and the stale label will be removed.

Otherwise this issue will be automatically closed in 28 days time.

[pp-mo]

I'm dismayed to see how this got forgotten, but I do suspect that we know what the problem is here, in which case I can say that help for it is on the way

Broadly, I think this data as loaded from netcdf needs a concatenate instead of a merge (because then they already have a time coordinate). It doesn't apply to PP because chopping data up into 2D 'fields' inevitably converts any time dimension into scalar times.

This is one of many longstanding problems with load/merge/concatenate which I'm happy we are now at long last trying to address !

See : #6168 and Iris 3.11 release highlights

Also upcoming : #5375
This should see some progress in the next few weeks as we are focussing on "Dragon tasks"' in December.