Creation of a utility to help recover raw data files that were not cleanly closed

[bieryAtFnal]

Usage: HDF5LIBS_TestRecoverFile [-R] <filename>

This app has two modes.

The first mode (without the "-R" command-line option), prints out information about the relevant HDF5 file-level Attributes in the file. It also prints out the values that will be used if/when the file is recovered.

The second mode (with the "-R" command-line option), sets necessary HDF5 Attributes in the file to appropriate values.

Re-running this app on a recovered file is possible, and this can be useful in verifying that the changes that were made were reasonable.

There are a couple of broken files in np04-srv-005:/data2/kurtMetadataTests/reference/ that can be used for testing this new app. If you use "cp -p ", that will preserve the original timestamp of these sample data file(s).

[wesketchum]

I've tested and the tests work as stated, so I think those look good. I'll approve given that.

One thing that came up though: I saw errors like this (that were caught appropriately I think):

HDF5-DIAG: Error detected in HDF5 (1.12.0) thread 0:
  #000: /tmp/root/spack-stage/spack-stage-hdf5-1.12.0-zliyqbximdzhoh4xppzbl5zoloa5gl5g/spack-src/src/H5A.c line 470 in H5Aopen(): unable to open attribute: 'source_id_path_map'
    major: Attribute
    minor: Can't open object
  #001: /tmp/root/spack-stage/spack-stage-hdf5-1.12.0-zliyqbximdzhoh4xppzbl5zoloa5gl5g/spack-src/src/H5VLcallback.c line 1128 in H5VL_attr_open(): attribute open failed
    major: Virtual Object Layer
    minor: Can't open object
  #002: /tmp/root/spack-stage/spack-stage-hdf5-1.12.0-zliyqbximdzhoh4xppzbl5zoloa5gl5g/spack-src/src/H5VLcallback.c line 1095 in H5VL__attr_open(): attribute open failed
    major: Virtual Object Layer
    minor: Can't open object
  #003: /tmp/root/spack-stage/spack-stage-hdf5-1.12.0-zliyqbximdzhoh4xppzbl5zoloa5gl5g/spack-src/src/H5VLnative_attr.c line 126 in H5VL__native_attr_open(): unable to open attribute: 'source_id_path_map'
    major: Attribute
    minor: Can't open object
  #004: /tmp/root/spack-stage/spack-stage-hdf5-1.12.0-zliyqbximdzhoh4xppzbl5zoloa5gl5g/spack-src/src/H5Aint.c line 435 in H5A__open(): unable to load attribute info from object header for attribute: 'source_id_path_map'
    major: Attribute
    minor: Can't open object
  #005: /tmp/root/spack-stage/spack-stage-hdf5-1.12.0-zliyqbximdzhoh4xppzbl5zoloa5gl5g/spack-src/src/H5Oattribute.c line 508 in H5O__attr_open_by_name(): can't locate attribute: 'source_id_path_map'
    major: Attribute
    minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.12.0) thread 0:
  #000: /tmp/root/spack-stage/spack-stage-hdf5-1.12.0-zliyqbximdzhoh4xppzbl5zoloa5gl5g/spack-src/src/H5A.c line 470 in H5Aopen(): unable to open attribute: 'fragment_type_source_id_map'
    major: Attribute
    minor: Can't open object
  #001: /tmp/root/spack-stage/spack-stage-hdf5-1.12.0-zliyqbximdzhoh4xppzbl5zoloa5gl5g/spack-src/src/H5VLcallback.c line 1128 in H5VL_attr_open(): attribute open failed
    major: Virtual Object Layer
    minor: Can't open object
  #002: /tmp/root/spack-stage/spack-stage-hdf5-1.12.0-zliyqbximdzhoh4xppzbl5zoloa5gl5g/spack-src/src/H5VLcallback.c line 1095 in H5VL__attr_open(): attribute open failed
    major: Virtual Object Layer
    minor: Can't open object
  #003: /tmp/root/spack-stage/spack-stage-hdf5-1.12.0-zliyqbximdzhoh4xppzbl5zoloa5gl5g/spack-src/src/H5VLnative_attr.c line 126 in H5VL__native_attr_open(): unable to open attribute: 'fragment_type_source_id_map'
    major: Attribute
    minor: Can't open object
  #004: /tmp/root/spack-stage/spack-stage-hdf5-1.12.0-zliyqbximdzhoh4xppzbl5zoloa5gl5g/spack-src/src/H5Aint.c line 435 in H5A__open(): unable to load attribute info from object header for attribute: 'fragment_type_source_id_map'
    major: Attribute
    minor: Can't open object
  #005: /tmp/root/spack-stage/spack-stage-hdf5-1.12.0-zliyqbximdzhoh4xppzbl5zoloa5gl5g/spack-src/src/H5Oattribute.c line 508 in H5O__attr_open_by_name(): can't locate attribute: 'fragment_type_source_id_map'
    major: Attribute
    minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.12.0) thread 0:
  #000: /tmp/root/spack-stage/spack-stage-hdf5-1.12.0-zliyqbximdzhoh4xppzbl5zoloa5gl5g/spack-src/src/H5A.c line 470 in H5Aopen(): unable to open attribute: 'subdetector_source_id_map'
    major: Attribute
    minor: Can't open object
  #001: /tmp/root/spack-stage/spack-stage-hdf5-1.12.0-zliyqbximdzhoh4xppzbl5zoloa5gl5g/spack-src/src/H5VLcallback.c line 1128 in H5VL_attr_open(): attribute open failed
    major: Virtual Object Layer
    minor: Can't open object
  #002: /tmp/root/spack-stage/spack-stage-hdf5-1.12.0-zliyqbximdzhoh4xppzbl5zoloa5gl5g/spack-src/src/H5VLcallback.c line 1095 in H5VL__attr_open(): attribute open failed
    major: Virtual Object Layer
    minor: Can't open object
  #003: /tmp/root/spack-stage/spack-stage-hdf5-1.12.0-zliyqbximdzhoh4xppzbl5zoloa5gl5g/spack-src/src/H5VLnative_attr.c line 126 in H5VL__native_attr_open(): unable to open attribute: 'subdetector_source_id_map'
    major: Attribute
    minor: Can't open object
  #004: /tmp/root/spack-stage/spack-stage-hdf5-1.12.0-zliyqbximdzhoh4xppzbl5zoloa5gl5g/spack-src/src/H5Aint.c line 435 in H5A__open(): unable to load attribute info from object header for attribute: 'subdetector_source_id_map'
    major: Attribute
    minor: Can't open object
  #005: /tmp/root/spack-stage/spack-stage-hdf5-1.12.0-zliyqbximdzhoh4xppzbl5zoloa5gl5g/spack-src/src/H5Oattribute.c line 508 in H5O__attr_open_by_name(): can't locate attribute: 'subdetector_source_id_map'
    major: Attribute
    minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.12.0) thread 0:
  #000: /tmp/root/spack-stage/spack-stage-hdf5-1.12.0-zliyqbximdzhoh4xppzbl5zoloa5gl5g/spack-src/src/H5D.c line 280 in H5Dopen2(): name parameter cannot be an empty string
    major: Invalid arguments to routine
    minor: Bad value

Is this expected for files needing recovery? If so, maybe this is a separate issue to try to better recover these attributes (if we think we need to)? Where 'try to recover' may be writing these at file open or something like that?

[bieryAtFnal]

Yes, I haven't looked very closely at the errors that we see from the last TriggerRecord in the un-closed PDS files, but now that you mention the details of those errors, it does seem like they may be recoverable. I ran hdf5_dump.py on one of the un-closed files, and it successfully printed out TriggerRecordHeader and FragmentHeader information for the last TriggerRecord in the file, so maybe all that is wrong with that TR is the missing Attribute(s) that are mentioned in the console messages listed above.
I'll file a separate Issue...