How do I modify a data value encoded in a reference dict?

[rsignell]

I've generated a dict of references for a NetCDF file, and I'd like to modify an encoded time value in the dict.

I generated the dict thusly:

import fsspec
from kerchunk.netCDF3 import NetCDF3ToZarr

nc_s3 = 's3://hycom-gofs-3pt1-reanalysis/2014/hycom_GLBv0.08_538_2014010112_t000.nc'
d = NetCDF3ToZarr(nc_s3, storage_options={"anon": True},
                  inline_threshold=400, version=2).translate()

which encodes the time value 112748.0 as

print(d['refs']['time/0'])
'base64:QP33wAAAAAA='

I want to specify the time value as 112751.0 (e.g. 3 hours later in this case), but I'm struggling with how to encode a floating point number the same way kerchunk is currently doing it.

As a test, I tried to replace the time value 112748.0 with the same value, expecting to get the same encoding.

import base64
import struct

def float_to_base64(number):
    # Pack the float into bytes
    packed = struct.pack('!f', number)
    
    # Encode the bytes to base64
    encoded = base64.b64encode(packed)
    # Convert bytes to string and return
    return encoded.decode('utf-8')

f'base64:{float_to_base64(112748.0)}''R9w2AA=='

returns

'base64:R9w2AA=='    

Clearly my string does not match the string kerchunk generated.

What am I doing wrong?

Note: I determined the time value was 112748.0 by running this code:

ds = xr.open_dataset(d, engine='kerchunk', chunks={}, drop_variables='tau', mask_and_scale=False,
                    decode_times=False, backend_kwargs=dict(storage_options=dict(
                    remote_protocol='s3', remote_options=dict(anon=True))))
ds.time.values

which produces:

array([122748.])

[martindurant]

The data actually has "dtype":">f8" - i.e., it is big endian and 64-bit; the former is super weird for floats, but there you go.

So changing to struct.pack('>d', number) produces

>>> float_to_base64(112748.0)
'QPuGwAAAAAA='

Alternative way

fs = fsspec.filesystem("reference", fo=d)
z = zarr.open(fs.get_mapper())
z.time[:] # original float value
z.time[:] += 3 # edit as required
fs.references["time/0"] # sets value as bytes
fs.save_json("out.json") # encodes as base64