latest HDF5 (latest environment) raises buffer error for writing large arrays

[bearecinos]

@dngoldberg, @jrmaddison

If I write some of the model diagnostics in hdf5 file format I get the buffer error:

*** buffer overflow detected ***: terminated

The problem comes up if write_diagnostics = true in the .toml configuration file.

The code fails on this line from the def gen_alpha(self, a_bgd=500.0, a_lb=1e2, a_ub=1e4): function.

The bottle neck of the issue, is with the HDF5 format, that is if output_var_format = 'h5' or output_var_format = 'all'.

And the lines where we write diagnostics output as HDF5 are here.

I'm not sure if this fails because we have reach the number of bytes that we are allow to write? Or maybe we are writing things wrong. Things seems to be find when we write the inversion output though, that is also written in hdf5 and that does not fail.

This how we write things in the function:

hdf5out = HDF5File(MPI.COMM_WORLD, str(outfname.with_suffix(".h5")), 'w')
hdf5out.write(outvar, name)
hdf5out.close()

For the inversion output the write things slightly different:

invout = HDF5File(mesh.mpi_comm(), str(outdir/invout_file), 'w')

invout.parameters.add("gamma_alpha", slvr.gamma_alpha)
invout.parameters.add("delta_alpha", slvr.delta_alpha)
invout.parameters.add("gamma_beta", slvr.gamma_beta)
invout.parameters.add("delta_beta", slvr.delta_beta)
invout.parameters.add("delta_beta_gnd", slvr.delta_beta_gnd)
invout.parameters.add("timestamp", str(datetime.datetime.now()))

invout.write(mdl.alpha, 'alpha')
invout.write(mdl.beta, 'beta')

Some thoughts on this will be good