Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DT-777] Filter by participant count #2733

Merged
merged 23 commits into from
Dec 2, 2024
+158 −39
Merged

[DT-777] Filter by participant count #2733

Show file tree
Hide file tree
Changes from 21 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
2cbedca
add filter for participant count
raejohanek Nov 21, 2024
45612b9
generalize FilterItemRange
raejohanek Nov 21, 2024
9b2edc9
Merge branch 'develop' into rj/dt-777-filter-participant-count
raejohanek Nov 21, 2024
e2d3e06
add unit test
raejohanek Nov 22, 2024
37606ba
feedback: update calculation of default values
raejohanek Nov 22, 2024
9f89de8
feedback: update numSelectedFilters -> anyFiltersSelected
raejohanek Nov 22, 2024
0f08b2c
simplify anyFiltersSelected
raejohanek Nov 22, 2024
8dd3cc4
Feedback: update filterHandler
raejohanek Nov 22, 2024
d03c0bf
Feedback: add validation
raejohanek Nov 22, 2024
e1383f1
Merge branch 'develop' into rj/dt-777-filter-participant-count
raejohanek Nov 22, 2024
d0556f8
try to fix test
raejohanek Nov 22, 2024
c033344
try to fix test
raejohanek Nov 22, 2024
4cc1ac8
try to fix test
raejohanek Nov 22, 2024
44e9f1c
try to fix test
raejohanek Nov 22, 2024
2417d78
fix tests?
raejohanek Nov 25, 2024
af1a501
fails locally, but just curious if it will pass on the server
raejohanek Nov 25, 2024
24ca3b1
revert last change
raejohanek Nov 25, 2024
c4ed6ee
reduce URLs and rename parameter
raejohanek Nov 25, 2024
2d3b3ab
remove debounce just to see
raejohanek Nov 25, 2024
851879e
add cy.clock and revert remove debounce
raejohanek Nov 25, 2024
455abe3
add cy.clock to existing tests!!
raejohanek Nov 26, 2024
ff293e5
pr feedback: simplify anyFiltersSelected and rename response alias in…
raejohanek Nov 26, 2024
79eaba9
pr feedback: update calls in test
raejohanek Dec 2, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 14 additions & 12 deletions cypress/component/DataSearch/dataset_search_filters.spec.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,18 +4,20 @@ import { mount } from 'cypress/react';
import React from 'react';
import DatasetFilterList from '../../../src/components/data_search/DatasetFilterList';

const duosUser = {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unused

isSigningOfficial: false,
};

describe('Data Library Filters', () => {
beforeEach(() => {
cy.initApplicationConfig();
});
// Intercept configuration calls
beforeEach(() => {
cy.initApplicationConfig();
});

it('Renders the data library filters', () => {
const props = { datasets: [], filters: [], filterHandler: () => {}, isFiltered: () => {}};
mount(<DatasetFilterList {...props} />);
cy.get('div').should('contain', 'Filters');
});
it('Renders the data library filters', () => {
const props = { datasets: [], filterHandler: () => {}, isFiltered: () => {}};
mount(<DatasetFilterList {...props} />);
cy.get('div').should('contain', 'Filters');
cy.get('div').should('contain', 'Access Type');
cy.get('div').should('contain', 'Data Use');
cy.get('div').should('contain', 'Data Access Committee');
cy.get('div').should('contain', 'Data Type');
cy.get('div').should('contain', 'Participant Count');
});
});
56 changes: 55 additions & 1 deletion cypress/component/DataSearch/dataset_search_table.spec.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,9 @@ const datasets = [
datasetId: 123456,
datasetIdentifier: `DUOS-123456`,
datasetName: 'Some Dataset 1',
participantCount: 100,
study: {
studyName: 'Some Study 1',
studyId: 1,
dataCustodianEmail: ['Some Data Custodian Email 1'],
}
Expand All @@ -23,9 +25,11 @@ const props = {

describe('Dataset Search Table tests', () => {

describe('Data library with three datasets', () => {
describe('Data library with one dataset footer tests', () => {
beforeEach(() => {
cy.initApplicationConfig();
cy.stub(TerraDataRepo, 'listSnapshotsByDatasetIds').returns({});
cy.clock();
mount(<DatasetSearchTable {...props} />);
});

Expand All @@ -38,6 +42,56 @@ describe('Dataset Search Table tests', () => {
cy.get('#header-checkbox').click();
cy.contains('1 dataset selected from 1 study');
});
});


describe('Data library filter by participant count tests', () => {

beforeEach(() => {
cy.initApplicationConfig();
cy.stub(TerraDataRepo, 'listSnapshotsByDatasetIds').returns({});
cy.clock();
});

function handler(request, searchText) {
if (JSON.stringify(request.body).includes(searchText)) {
request.reply(['filtered']);
} else {
request.reply([]);
}
}


it('When a participant count filter is applied the query is updated', () => {
cy.intercept(
{method: 'POST', url: '**/search/index'}, (req) => {
return handler(req, '{"range":{"participantCount":{"gte":null,"lte":50}}}');
}).as('searchIndex1');
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it required that searchIndex1 must be different than searchIndex2 in the other test? I would expect the alias names to be independent across tests so the same name could be used for both.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good question, no I don't think so I just changed them to detangle as part of the debugging effort but I can change them back now

mount(<DatasetSearchTable {...props} />);
// first clear the default value (100), without clearing first, type('50') would result in input of 10050
cy.get('#participantCountMax-range-input').clear().type('50');
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cypress recommends not chaining calls after clear. It's a little more verbose, but safer to:

      const range = cy.get('#participantCountMax-range-input');
      range.clear();
      range.type('50');

cy.tick(150);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cypress recommends that there be a clock command that precedes the tick command: https://docs.cypress.io/api/commands/tick

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I use the clock command in the before each method, but I could move it into each individual test

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, I did miss that. No need if it's there, that should cover it.

// this api call should have had a request that contained the searchText
cy.wait('@searchIndex1').then((response) => {
expect(response.response.body[0]).to.equal('filtered');
});
cy.get('@searchIndex1.all').should('have.length', 1);

});

it('When an invalid participant count filter is applied the query represents the default value', () => {
cy.intercept({method: 'POST', url: '**/search/index'}, (req) => {
// when non-numeric input is entered, the default value (in this case, 100) is used
return handler(req, '{"range":{"participantCount":{"gte":100,"lte":null}}}');
}).as('searchIndex2');
mount(<DatasetSearchTable {...props} />);
cy.get('#participantCountMin-range-input').type('test');
cy.tick(150);
cy.wait('@searchIndex2').then((response) => {
expect(response.response.body[0]).to.equal('filtered');
});
cy.get('@searchIndex2.all').should('have.length', 1);
});

});
});
60 changes: 46 additions & 14 deletions src/components/data_search/DatasetFilterList.jsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,10 @@ import ListItemButton from '@mui/material/ListItemButton';
import ListItemIcon from '@mui/material/ListItemIcon';
import ListItemText from '@mui/material/ListItemText';
import Divider from '@mui/material/Divider';
import { Button, Typography } from '@mui/material';
import { Button, TextField, Typography } from '@mui/material';
import { Checkbox } from '@mui/material';
import {flatten, uniq, compact, orderBy} from 'lodash';
import {getAccessManagementSummary} from '../../types/model';
import { flatten, uniq, compact, orderBy } from 'lodash';
import { getAccessManagementSummary } from '../../types/model';

export const FilterItemHeader = (props) => {
const { title, headerStyle = { fontFamily: 'Montserrat', fontWeight: '600', marginTop: '1em' } } = props;
Expand All @@ -21,20 +21,20 @@ export const FilterItemHeader = (props) => {
};

export const FilterItemList = (props) => {
const { category, datasets, filter, filterHandler, isFiltered, filterNameFn, filterDisplayFn } = props;
const { category, filter, filterHandler, isFiltered, filterNameFn, filterDisplayFn } = props;
return (
<List sx={{ margin: '-0.5em -0.5em' }}>
{
filter.map((filter) => {
const filterName = filterNameFn(filter);
filter.map((filterOption) => {
const filterName = filterNameFn(filterOption);
return (
<ListItem disablePadding key={filter}>
<ListItemButton sx={{ padding: '0' }} onClick={(event) => filterHandler(event, datasets, category, filter)}>
<ListItem disablePadding key={filterOption}>
<ListItemButton sx={{ padding: '0' }} onClick={() => filterHandler(category, filterOption)}>
<ListItemIcon>
<Checkbox checked={isFiltered(filter, category)} />
<Checkbox checked={isFiltered(filterOption, category)} />
</ListItemIcon>
<ListItemText sx={{ fontFamily: 'Montserrat', transform: 'scale(1.2)' }}>
{filterDisplayFn ? filterDisplayFn(filter) : filterName}
{filterDisplayFn ? filterDisplayFn(filterOption) : filterName}
</ListItemText>
</ListItemButton>
</ListItem>
Expand All @@ -45,14 +45,37 @@ export const FilterItemList = (props) => {
);
};

export const FilterItemRange = (props) => {
const { min, max, minCategory, maxCategory, filterHandler } = props;
const getValue = (val, defaultVal) => isNaN(Number(val)) ? defaultVal : Number(val);
return (
<Box key={minCategory + '-' + maxCategory} sx={{ display: 'flex', flexDirection: 'row', alignItems: 'center' }}>
<TextField id={minCategory + '-range-input'} size='small' margin='dense' variant='outlined' defaultValue={min}
helperText={'minimum'}
FormHelperTextProps={{style: { transform: 'scale(1.5)' }}}
onChange={(event) => filterHandler(minCategory, getValue(event.target.value, min))}/>
<Box padding={'0rem 1rem 1rem'}> - </Box>
<TextField id={maxCategory + '-range-input'} size='small' margin='dense' variant='outlined' defaultValue={max}
helperText={'maximum'}
FormHelperTextProps={{style: {transform: 'scale(1.5)'}}}
onChange={(event) => filterHandler(maxCategory, getValue(event.target.value, max))}
/>
</Box>
);
};

export const DatasetFilterList = (props) => {
const { datasets, filters, filterHandler, isFiltered, onClear } = props;
const { datasets, filterHandler, isFiltered, onClear } = props;

const accessManagementFilters = uniq(compact(datasets.map((dataset) => dataset.accessManagement)));
const dataUseFilters = uniq(compact(flatten(datasets.map((dataset) => dataset.dataUse?.primary))).map((dataUse) => dataUse.code));
const dataTypeFilters = uniq(flatten(datasets.map((dataset) => dataset.study.dataTypes)));
const dacFilters = orderBy(uniq(compact(datasets.map((dataset) => dataset.dac?.dacName))), (dac) => dac.toLowerCase(), 'asc');

const defaultValues = datasets.reduce((acc, dataset) => {
return {
max: Math.max(acc.max, dataset.participantCount ? dataset.participantCount : 0),
min: Math.min(acc.min, dataset.participantCount ? dataset.participantCount : Infinity) };
}, {max: 0, min: Infinity});
return (
<Box sx={{ bgcolor: 'background.paper' }}>
<Box sx={{ display: 'flex', justifyContent: 'space-between', alignItems: 'center' }}>
Expand Down Expand Up @@ -101,15 +124,24 @@ export const DatasetFilterList = (props) => {
isFiltered={isFiltered}
filterNameFn={(filter) => filter}
/>
<FilterItemHeader title="Data Type" />
<FilterItemHeader title='Data Type' />
<FilterItemList
category="dataType"
category='dataType'
datasets={datasets}
filter={dataTypeFilters}
filterHandler={filterHandler}
isFiltered={isFiltered}
filterNameFn={(filter) => filter}
/>
<FilterItemHeader title='Participant Count' />
<FilterItemRange
min={defaultValues.min}
max={defaultValues.max}
minCategory='participantCountMin'
rjohanek marked this conversation as resolved.
Show resolved Hide resolved
maxCategory='participantCountMax'
datasets={datasets}
filterHandler={filterHandler}
/>
</Box>
);
};
Expand Down
50 changes: 38 additions & 12 deletions src/components/data_search/DatasetSearchTable.jsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@ import Tabs from '@mui/material/Tabs';
import useOnMount from '@mui/utils/useOnMount';
import * as React from 'react';
import { Box, Button } from '@mui/material';
import { useEffect, useRef, useState } from 'react';
import { isEmpty } from 'lodash';
import {useEffect, useRef, useState} from 'react';
import { isArray, isEmpty } from 'lodash';
import { TerraDataRepo } from '../../libs/ajax/TerraDataRepo';
import { DatasetSearchTableDisplay } from './DatasetSearchTableDisplay';
import { datasetSearchTableTabs } from './DatasetSearchTableConstants';
Expand Down Expand Up @@ -37,7 +37,9 @@ const defaultFilters = {
accessManagement: [],
dataUse: [],
dataType: [],
dac: []
dac: [],
participantCountMin: null,
participantCountMax: null,
};

export const DatasetSearchTable = (props) => {
Expand All @@ -49,8 +51,17 @@ export const DatasetSearchTable = (props) => {
const [selectedTable, setSelectedTable] = useState(datasetSearchTableTabs.study);
const [searchTerm, setSearchTerm] = useState('');

const isFiltered = (filter, category) => (filters[category]).indexOf(filter) > -1;
const numSelectedFilters = (filters) => Object.values(filters).reduce((sum, array) => sum + array.length, 0);
const isFilteredArray = (filter, category) => (filters[category]).indexOf(filter) > -1;

const anyFiltersSelected = (filters) => {
return Object.values(filters).some((filter) => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For a single line lambda, you can omit the { return }

Suggested change
const anyFiltersSelected = (filters) => {
return Object.values(filters).some((filter) => {
const anyFiltersSelected = (filters) =>
Object.values(filters).some((filter) => {

I know in terra-ui this was the standard, not sure if duos-ui has an opinion about this.

if (isArray(filter)) {
return filter.length > 0;
} else {
return filter !== null;
}
pshapiro4broad marked this conversation as resolved.
Show resolved Hide resolved
});
};

const getExportableDatasets = async (datasets) => {
// Note the dataset identifier is in each sub-table row.
Expand Down Expand Up @@ -108,7 +119,7 @@ export const DatasetSearchTable = (props) => {
}

let filterQuery = {};
if (numSelectedFilters(filters) > 0) {
if (anyFiltersSelected(filters)) {
const filterTerms = [];

filterTerms.push({
Expand Down Expand Up @@ -155,6 +166,15 @@ export const DatasetSearchTable = (props) => {
}
});

filterTerms.push({
'range': {
'participantCount': {
'gte': filters.participantCountMin,
'lte': filters.participantCountMax,
rjohanek marked this conversation as resolved.
Show resolved Hide resolved
}
}
});

if (filterTerms.length > 0) {
filterQuery = [
{
Expand All @@ -179,13 +199,19 @@ export const DatasetSearchTable = (props) => {
};
};

const filterHandler = (event, data, category, filter) => {
var newFilters = _.clone(filters);
if (!isFiltered(filter, category) && filter !== '') {
newFilters[category] = filters[category].concat(filter);
const filterHandler = (category, filter) => {
let newFilter;
if (isArray(filters[category])) {
if (!isFilteredArray(filter, category) && filter !== '') {
newFilter = filters[category].concat(filter);
} else {
newFilter = filters[category].filter((f) => f !== filter);
}
} else {
newFilters[category] = filters[category].filter((f) => f !== filter);
newFilter = filter;
}
const newFilters = _.clone(filters);
newFilters[category] = newFilter;
setFilters(newFilters);
};

Expand Down Expand Up @@ -280,7 +306,7 @@ export const DatasetSearchTable = (props) => {
</Box>
<Box sx={{display: 'flex', flexDirection: 'row', paddingTop: '2em'}}>
<Box sx={{width: '14%', padding: '0 1em'}}>
<DatasetFilterList datasets={datasets} filters={filters} filterHandler={filterHandler} isFiltered={isFiltered} onClear={() => setFilters(defaultFilters)}/>
<DatasetFilterList datasets={datasets} filterHandler={filterHandler} isFiltered={isFilteredArray} onClear={() => setFilters(defaultFilters)}/>
</Box>
<Box sx={{width: '85%', padding: '0 1em'}}>
{(() => {
Expand Down
Loading