Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support visualizing enum datasets and signals #1688

Merged
merged 6 commits into from
Aug 1, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file modified cypress/snapshots/app.cy.ts/auximage.snap.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified cypress/snapshots/app.cy.ts/auxspectrum.snap.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified cypress/snapshots/app.cy.ts/bgr_image.snap.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified cypress/snapshots/app.cy.ts/compound_1D.snap.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified cypress/snapshots/app.cy.ts/heatmap_2D.snap.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified cypress/snapshots/app.cy.ts/heatmap_2D_complex.snap.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified cypress/snapshots/app.cy.ts/heatmap_2D_inverted_cmap.snap.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified cypress/snapshots/app.cy.ts/heatmap_4d_default.snap.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified cypress/snapshots/app.cy.ts/heatmap_4d_remapped.snap.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified cypress/snapshots/app.cy.ts/heatmap_4d_sliced.snap.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified cypress/snapshots/app.cy.ts/heatmap_4d_zeros.snap.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified cypress/snapshots/app.cy.ts/heatmap_domain.snap.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified cypress/snapshots/app.cy.ts/line_1D.snap.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified cypress/snapshots/app.cy.ts/line_complex_1D.snap.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified cypress/snapshots/app.cy.ts/logspectrum.snap.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified cypress/snapshots/app.cy.ts/matrix_1D.snap.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified cypress/snapshots/app.cy.ts/nximage.snap.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified cypress/snapshots/app.cy.ts/nximage_complex_2d.snap.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified cypress/snapshots/app.cy.ts/nxline.snap.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified cypress/snapshots/app.cy.ts/nxline_complex_2d_aux.snap.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified cypress/snapshots/app.cy.ts/nxrgb.snap.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified cypress/snapshots/app.cy.ts/nxscatter.snap.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified cypress/snapshots/app.cy.ts/rgb_image.snap.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
27 changes: 27 additions & 0 deletions packages/app/src/__tests__/CorePack.test.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,22 @@ test('visualize 1D dataset as matrix', async () => {
expect(screen.getByText('3.610e+2')).toBeVisible();
});

test('visualize 1D boolean dataset', async () => {
await renderApp('/nD_datasets/oneD_bool');

expect(getVisTabs()).toEqual([Vis.Matrix, Vis.Line]);
expect(getSelectedVisTab()).toBe(Vis.Line);
expect(screen.getByRole('figure', { name: 'oneD_bool' })).toBeVisible();
});

test('visualize 1D enum dataset', async () => {
await renderApp('/nD_datasets/oneD_enum');

expect(getVisTabs()).toEqual([Vis.Matrix, Vis.Line]);
expect(getSelectedVisTab()).toBe(Vis.Line);
expect(screen.getByRole('figure', { name: 'oneD_enum' })).toBeVisible();
});

test('visualize 1D complex dataset', async () => {
await renderApp('/nD_datasets/oneD_cplx');

Expand Down Expand Up @@ -119,6 +135,17 @@ test('visualize 2D boolean dataset', async () => {
expect(within(figure).getByText('1e+0')).toBeVisible(); // color bar limit
});

test('visualize 2D enum dataset', async () => {
await renderApp('/nD_datasets/twoD_enum');

expect(getVisTabs()).toEqual([Vis.Matrix, Vis.Line, Vis.Heatmap]);
expect(getSelectedVisTab()).toBe(Vis.Heatmap);

const figure = screen.getByRole('figure', { name: 'twoD_enum' });
expect(figure).toBeVisible();
expect(within(figure).getByText('2e+0')).toBeVisible(); // color bar limit
});

test('visualize 2D complex dataset', async () => {
const { user } = await renderApp('/nD_datasets/twoD_cplx');

Expand Down
20 changes: 19 additions & 1 deletion packages/app/src/__tests__/NexusPack.test.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,22 @@ test('visualize NXdata group with old-style signal', async () => {
).toBeVisible();
});

test('visualize NXdata group with boolean signal', async () => {
await renderApp('/nexus_entry/numeric-like/bool');
expect(getVisTabs()).toEqual([NexusVis.NxSpectrum, NexusVis.NxImage]);
expect(
screen.getByRole('figure', { name: 'twoD_bool' }), // name of dataset with `signal` attribute
).toBeVisible();
});

test('visualize NXdata group with enum signal', async () => {
await renderApp('/nexus_entry/numeric-like/enum');
expect(getVisTabs()).toEqual([NexusVis.NxSpectrum, NexusVis.NxImage]);
expect(
screen.getByRole('figure', { name: 'twoD_enum' }), // name of dataset with `signal` attribute
).toBeVisible();
});

test('visualize group with `default` attribute', async () => {
// NXroot with relative path to NXentry group with relative path to NXdata group with 2D signal
const { selectExplorerNode } = await renderApp();
Expand Down Expand Up @@ -177,7 +193,9 @@ test('show error/fallback for malformed NeXus entity', async () => {
// Type of signal dataset is not numeric
await selectExplorerNode('signal_not_numeric');
expect(
screen.getByText('Expected dataset to have numeric or complex type'),
screen.getByText(
'Expected dataset to have numeric, boolean, enum or complex type',
),
).toBeVisible();
errorSpy.mockClear();

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -1720,8 +1720,8 @@ exports[`test file matches snapshot 1`] = `
},
"class": "Enumeration",
"mapping": {
"A": 0,
"B": 1,
"0": "A",
"1": "B",
loichuder marked this conversation as resolved.
Show resolved Hide resolved
},
},
"value": 1,
Expand Down Expand Up @@ -1753,12 +1753,119 @@ exports[`test file matches snapshot 1`] = `
},
"class": "Enumeration",
"mapping": {
"A": 256,
"B": 257,
"256": "A",
"257": "B",
},
},
"value": 256,
},
{
"name": "enum_1D",
"rawType": {
"base": {
"class": 0,
"dtype": "|u1",
"order": 0,
"sign": 0,
"size": 1,
},
"class": 8,
"dtype": "|u1",
"members": {
"A": 0,
"B": 1,
"C": 2,
},
"size": 1,
},
"shape": [
10,
],
"type": {
"base": {
"class": "Integer (unsigned)",
"endianness": "little-endian",
"size": 8,
},
"class": "Enumeration",
"mapping": {
"0": "A",
"1": "B",
"2": "C",
},
},
"value": Uint8Array [
loichuder marked this conversation as resolved.
Show resolved Hide resolved
0,
2,
2,
1,
1,
0,
2,
2,
1,
1,
],
},
{
"name": "enum_2D",
"rawType": {
"base": {
"class": 0,
"dtype": "|u1",
"order": 0,
"sign": 0,
"size": 1,
},
"class": 8,
"dtype": "|u1",
"members": {
"A": 0,
"B": 1,
"C": 2,
},
"size": 1,
},
"shape": [
2,
10,
],
"type": {
"base": {
"class": "Integer (unsigned)",
"endianness": "little-endian",
"size": 8,
},
"class": "Enumeration",
"mapping": {
"0": "A",
"1": "B",
"2": "C",
},
},
"value": Uint8Array [
0,
2,
2,
1,
1,
0,
2,
2,
1,
1,
2,
0,
1,
1,
0,
2,
2,
2,
2,
1,
],
},
{
"name": "vlen_int8_scalar",
"rawType": {
Expand Down
8 changes: 2 additions & 6 deletions packages/app/src/providers/hsds/utils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,13 +17,11 @@ import type {
} from '@h5web/shared/hdf5-models';
import { DTypeClass } from '@h5web/shared/hdf5-models';
import {
boolType,
compoundType,
cplxType,
enumType,
enumOrBoolType,
floatType,
intType,
isBoolEnumType,
strType,
uintType,
unknownType,
Expand Down Expand Up @@ -123,9 +121,7 @@ function convertHsdsCompoundType(
function convertHsdsEnumType(hsdsType: HsdsEnumType): EnumType | BooleanType {
const { base, mapping } = hsdsType;
assertHsdsNumericType(base);

const type = enumType(convertHsdsNumericType(base), mapping);
return isBoolEnumType(type) ? boolType() : type; // booleans stored as enums by h5py
return enumOrBoolType(convertHsdsNumericType(base), mapping);
}

export function convertHsdsType(hsdsType: HsdsType): DType {
Expand Down
18 changes: 16 additions & 2 deletions packages/app/src/providers/mock/mock-file.ts
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,8 @@ const PNG_RED_DOT = new Uint8Array([
0x45, 0x4e, 0x44, 0xae, 0x42, 0x60, 0x82,
]);

const ENUM_MAPPING = { FOO: 0, BAR: 1, BAZ: 2 };

export function makeMockFile(): GroupWithChildren {
return nxGroup('source.h5', 'NXroot', {
isRoot: true,
Expand Down Expand Up @@ -87,10 +89,10 @@ export function makeMockFile(): GroupWithChildren {
],
}),
scalar('scalar_enum', 2, {
type: enumType(uintType(8), { FOO: 0, BAR: 1, BAZ: 2 }),
type: enumType(uintType(8), ENUM_MAPPING),
attributes: [
scalarAttr('attr', 2, {
type: enumType(uintType(8), { FOO: 0, BAR: 1, BAZ: 2 }),
type: enumType(uintType(8), ENUM_MAPPING),
}),
],
}),
Expand All @@ -117,6 +119,7 @@ export function makeMockFile(): GroupWithChildren {
}),
}),
array('oneD_bool'),
array('oneD_enum', { type: enumType(uintType(8), ENUM_MAPPING) }),
array('twoD'),
array('twoD_cplx'),
array('twoD_compound', {
Expand All @@ -129,6 +132,7 @@ export function makeMockFile(): GroupWithChildren {
}),
}),
array('twoD_bool'),
array('twoD_enum', { type: enumType(uintType(8), ENUM_MAPPING) }),
array('threeD'),
array('threeD_bool'),
array('threeD_cplx'),
Expand Down Expand Up @@ -268,6 +272,16 @@ export function makeMockFile(): GroupWithChildren {
}),
],
}),
nxGroup('numeric-like', 'NXprocess', {
children: [
nxData('bool', { signal: array('twoD_bool') }),
nxData('enum', {
signal: array('twoD_enum', {
type: enumType(uintType(8), ENUM_MAPPING),
}),
}),
],
}),
],
}),
nxGroup('nexus_note', 'NXnote', {
Expand Down
2 changes: 1 addition & 1 deletion packages/app/src/providers/models.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@ import type {
} from '@h5web/shared/hdf5-models';
import type { FetchStore } from '@h5web/shared/react-suspense-fetch';

import type { ImageAttribute } from '../vis-packs/core/models';
import type { NxAttribute } from '../vis-packs/nexus/models';

export type EntitiesStore = FetchStore<string, ProvidedEntity>;
Expand All @@ -23,6 +22,7 @@ export interface AttrValuesStore extends FetchStore<Entity, AttributeValues> {
getSingle: (entity: Entity, attrName: AttrName) => unknown;
}

export type ImageAttribute = 'CLASS' | 'IMAGE_SUBCLASS';
export type AttrName = NxAttribute | ImageAttribute | '_FillValue';

export type ExportFormat = 'json' | 'csv' | 'npy' | 'tiff';
Expand Down
8 changes: 6 additions & 2 deletions packages/app/src/providers/utils.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { isNumericType } from '@h5web/shared/guards';
import { isEnumType, isNumericType } from '@h5web/shared/guards';
import type {
ArrayShape,
Dataset,
Expand Down Expand Up @@ -33,11 +33,15 @@ export async function handleAxiosError<T>(
}

export function typedArrayFromDType(dtype: DType) {
/* Adapted from https://github.com/ludwigschubert/js-numpy-parser/blob/v1.2.3/src/main.js#L116 */
if (isEnumType(dtype)) {
return typedArrayFromDType(dtype.base);
loichuder marked this conversation as resolved.
Show resolved Hide resolved
}

if (!isNumericType(dtype)) {
return undefined;
}

/* Adapted from https://github.com/ludwigschubert/js-numpy-parser/blob/v1.2.3/src/main.js#L116 */
const { class: dtypeClass, size } = dtype;

if (dtypeClass === DTypeClass.Integer) {
Expand Down
14 changes: 10 additions & 4 deletions packages/app/src/vis-packs/core/matrix/utils.ts
Original file line number Diff line number Diff line change
@@ -1,17 +1,19 @@
import { Notation } from '@h5web/lib';
import { isComplexType, isNumericType } from '@h5web/shared/guards';
import { isComplexType, isEnumType, isNumericType } from '@h5web/shared/guards';
import type {
ComplexType,
NumericType,
PrintableCompoundType,
PrintableType,
} from '@h5web/shared/hdf5-models';
import { DTypeClass } from '@h5web/shared/hdf5-models';
import { createComplexFormatter } from '@h5web/shared/vis-utils';
import type { ValueFormatter } from '@h5web/shared/vis-models';
import {
createComplexFormatter,
createEnumFormatter,
} from '@h5web/shared/vis-utils';
import { format } from 'd3-format';

import type { ValueFormatter } from '../models';

export function createNumericFormatter(
notation: Notation,
): ValueFormatter<NumericType> {
Expand Down Expand Up @@ -48,6 +50,10 @@ export function getFormatter(
return createNumericFormatter(notation);
}

if (isEnumType(type)) {
return createEnumFormatter(type.mapping);
}

return (val) => (val as string | boolean).toString();
}

Expand Down
5 changes: 0 additions & 5 deletions packages/app/src/vis-packs/core/models.ts

This file was deleted.

17 changes: 11 additions & 6 deletions packages/app/src/vis-packs/core/scalar/utils.ts
Original file line number Diff line number Diff line change
@@ -1,19 +1,24 @@
import { hasComplexType } from '@h5web/shared/guards';
import { hasComplexType, hasEnumType } from '@h5web/shared/guards';
import type {
ArrayShape,
Dataset,
H5WebComplex,
PrintableType,
} from '@h5web/shared/hdf5-models';
import { formatScalarComplex } from '@h5web/shared/vis-utils';

import type { ValueFormatter } from '../models';
import type { ValueFormatter } from '@h5web/shared/vis-models';
import {
createEnumFormatter,
formatScalarComplex,
} from '@h5web/shared/vis-utils';

export function getFormatter(
dataset: Dataset<ArrayShape, PrintableType>,
): ValueFormatter<PrintableType> {
if (hasComplexType(dataset)) {
return (val) => formatScalarComplex(val as H5WebComplex);
return formatScalarComplex;
}

if (hasEnumType(dataset)) {
return createEnumFormatter(dataset.type.mapping);
}

return (val) => (val as number | string | boolean).toString();
Expand Down
Loading