Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Float16 Datasets not supported #335

Open
G-nn-r opened this issue Oct 1, 2024 · 1 comment
Open

Float16 Datasets not supported #335

G-nn-r opened this issue Oct 1, 2024 · 1 comment
Assignees
@byrnHDF
Labels
Component - HDFView Improvements to the visual interface layer Priority - 1. High 🔼 These are important issues that should be resolved in the next release Type - Improvement Improvements that don't add a new feature or functionality

Comments

@G-nn-r
Copy link

G-nn-r commented Oct 1, 2024 •
edited
Loading

While it is very nice to see that float16 values (e.g., as object attributes) are finally supported since 3.3.2, unfortunately float16 datasets are not supported yet.

Example code to create a malicious h5 with Python:

import h5py
import numpy as np

with h5py.File(r"C:\temp\hdf5_test.h5", mode='w') as f:
    dset16 = f.create_dataset("dataset_float16", data=np.eye(5, dtype=np.float16))  # this cannot be opened in HDFView
    dset32 = f.create_dataset("dataset_float32", data=np.eye(5, dtype=np.float32))  # this can be opened in HDFView

Error messages when trying to open the dataset

For opening as a spreadsheet (table):

An error occured while loading data for the table:
failed to read scalar dataset: Inappropriate type

image

The bottom of the HDFView windows says this:

 *** Error: unable to load table data ***

The log shows this message:

java.lang.Exception: failed to read scalar dataset: Inappropriate type
        at hdf.object.h5.H5ScalarDS.read(H5ScalarDS.java:871)
        at hdf.view.dialog.DataOptionDialog$PreviewNavigator.createPreviewImage(DataOptionDialog.java:1789)
        at hdf.view.dialog.DataOptionDialog$PreviewNavigator.<init>(DataOptionDialog.java:1627)
        at hdf.view.dialog.DataOptionDialog.createScalarDSContents(DataOptionDialog.java:1149)
        at hdf.view.dialog.DataOptionDialog.open(DataOptionDialog.java:230)
        at hdf.view.TreeView.DefaultTreeView.showDataContent(DefaultTreeView.java:2945)
        at hdf.view.MetaDataView.DefaultDatasetMetaDataView$1.widgetSelected(DefaultDatasetMetaDataView.java:302)
        at swt/org.eclipse.swt.widgets.TypedListener.handleEvent(TypedListener.java:252)
        at swt/org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:89)
        at swt/org.eclipse.swt.widgets.Display.sendEvent(Display.java:4256)
        at swt/org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1066)
        at swt/org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:4054)
        at swt/org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3642)
        at hdf.view.HDFView.runMainWindow(HDFView.java:438)
        at hdf.view.HDFView$41.run(HDFView.java:2822)
        at swt/org.eclipse.swt.widgets.Synchronizer.syncExec(Synchronizer.java:183)
        at swt/org.eclipse.swt.widgets.Display.syncExec(Display.java:4785)
        at hdf.view.HDFView.main(HDFView.java:2812)
Caused by: java.lang.Exception: Inappropriate type
        at hdf.object.h5.H5ScalarDS.scalarDatasetCommonIO(H5ScalarDS.java:1021)
        at hdf.object.h5.H5ScalarDS.read(H5ScalarDS.java:867)
        ... 17 more
Caused by: hdf.hdf5lib.exceptions.HDF5FunctionArgumentException: Inappropriate type
        at [email protected]/hdf.hdf5lib.H5.H5Dread_short(Native Method)
        at [email protected]/hdf.hdf5lib.H5.H5Dread(H5.java:2986)
        at [email protected]/hdf.hdf5lib.H5.H5Dread(H5.java:2928)
        at hdf.object.h5.H5ScalarDS.scalarDatasetCommonIO(H5ScalarDS.java:1010)
        ... 18 more

For opening as an image

Select
ImageView:

image

The bottom of the HDFView windows says this:

 *** Loading image failed - dataset_float16 ***

The log shows this error (seems to be identical to the one above):

java.lang.Exception: failed to read scalar dataset: Inappropriate type
        at hdf.object.h5.H5ScalarDS.read(H5ScalarDS.java:871)
        at hdf.view.dialog.DataOptionDialog$PreviewNavigator.createPreviewImage(DataOptionDialog.java:1789)
        at hdf.view.dialog.DataOptionDialog$PreviewNavigator.<init>(DataOptionDialog.java:1627)
        at hdf.view.dialog.DataOptionDialog.createScalarDSContents(DataOptionDialog.java:1149)
        at hdf.view.dialog.DataOptionDialog.open(DataOptionDialog.java:230)
        at hdf.view.TreeView.DefaultTreeView.showDataContent(DefaultTreeView.java:2945)
        at hdf.view.MetaDataView.DefaultDatasetMetaDataView$1.widgetSelected(DefaultDatasetMetaDataView.java:302)
        at swt/org.eclipse.swt.widgets.TypedListener.handleEvent(TypedListener.java:252)
        at swt/org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:89)
        at swt/org.eclipse.swt.widgets.Display.sendEvent(Display.java:4256)
        at swt/org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1066)
        at swt/org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:4054)
        at swt/org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3642)
        at hdf.view.HDFView.runMainWindow(HDFView.java:438)
        at hdf.view.HDFView$41.run(HDFView.java:2822)
        at swt/org.eclipse.swt.widgets.Synchronizer.syncExec(Synchronizer.java:183)
        at swt/org.eclipse.swt.widgets.Display.syncExec(Display.java:4785)
        at hdf.view.HDFView.main(HDFView.java:2812)
Caused by: java.lang.Exception: Inappropriate type
        at hdf.object.h5.H5ScalarDS.scalarDatasetCommonIO(H5ScalarDS.java:1021)
        at hdf.object.h5.H5ScalarDS.read(H5ScalarDS.java:867)
        ... 17 more
Caused by: hdf.hdf5lib.exceptions.HDF5FunctionArgumentException: Inappropriate type
        at [email protected]/hdf.hdf5lib.H5.H5Dread_short(Native Method)
        at [email protected]/hdf.hdf5lib.H5.H5Dread(H5.java:2986)
        at [email protected]/hdf.hdf5lib.H5.H5Dread(H5.java:2928)
        at hdf.object.h5.H5ScalarDS.scalarDatasetCommonIO(H5ScalarDS.java:1010)
        ... 18 more

H5Dump

h5dump displays the contents correctly:

HDF5 "hdf5_test.h5" {
GROUP "/" {
   DATASET "dataset_float16" {
      DATATYPE  16-bit little-endian floating-point 16-bit precision
      DATASPACE  SIMPLE { ( 5, 5 ) / ( 5, 5 ) }
      DATA {
      (0,0): 1, 0, 0, 0, 0,
      (1,0): 0, 1, 0, 0, 0,
      (2,0): 0, 0, 1, 0, 0,
      (3,0): 0, 0, 0, 1, 0,
      (4,0): 0, 0, 0, 0, 1
      }
   }
   DATASET "dataset_float32" {
      DATATYPE  H5T_IEEE_F32LE
      DATASPACE  SIMPLE { ( 5, 5 ) / ( 5, 5 ) }
      DATA {
      (0,0): 1, 0, 0, 0, 0,
      (1,0): 0, 1, 0, 0, 0,
      (2,0): 0, 0, 1, 0, 0,
      (3,0): 0, 0, 0, 1, 0,
      (4,0): 0, 0, 0, 0, 1
      }
   }
}
}

Tested on

  • Platform: Windows 10
  • HDFView Versions: 3.3.2 and 99.99.99
@derobins derobins added Type - Improvement Improvements that don't add a new feature or functionality Component - HDFView Improvements to the visual interface layer Priority - 1. High 🔼 These are important issues that should be resolved in the next release labels Oct 1, 2024
@G-nn-r G-nn-r mentioned this issue Nov 11, 2024
Closed
@jhendersonHDF
Copy link
Collaborator

Hi @G-nn-r,

I believe this is an artifact of how HDFView was built and released on Windows specifically. Viewing float16 datasets works for me on Linux, for example, but I believe the MSVC version used to build HDF5 for the Windows release does not support the _Float16 datatype. This causes the Windows build to run into problems when trying to create a native datatype for reading the data in. We'll have this fixed for the next release of HDFView. Also, the Intel OneAPI compiler does support _Float16 on Windows, so it may be worth looking at having a separate build using it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@byrnHDF byrnHDF

Labels
Component - HDFView Improvements to the visual interface layer Priority - 1. High 🔼 These are important issues that should be resolved in the next release Type - Improvement Improvements that don't add a new feature or functionality
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@derobins @jhendersonHDF @byrnHDF @G-nn-r