Investigate exactly why HDF file locking is not working as expected #6774
Description
#6760 implements a workaround to a non-deterministic problem with HDF file locking. It is uncomfortable that we have a bug we cannot fully explain - it needs further investigation.
The leading hypothesis is that HDF is taking longer to release its file lock than Python is taking to release the Iris-level locks. Testing this will require using lower-level scripts (C scripts?) to experimentally eliminate the Python factor.
The problem appears file-system-specific. Testing should be performed with the files being saved to a Lustre file system.