Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DT-777] Filter by participant count #2733

Merged
merged 23 commits into from
Dec 2, 2024
Merged

[DT-777] Filter by participant count #2733

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
2cbedca
add filter for participant count
raejohanek Nov 21, 2024
45612b9
generalize FilterItemRange
raejohanek Nov 21, 2024
9b2edc9
Merge branch 'develop' into rj/dt-777-filter-participant-count
raejohanek Nov 21, 2024
e2d3e06
add unit test
raejohanek Nov 22, 2024
37606ba
feedback: update calculation of default values
raejohanek Nov 22, 2024
9f89de8
feedback: update numSelectedFilters -> anyFiltersSelected
raejohanek Nov 22, 2024
0f08b2c
simplify anyFiltersSelected
raejohanek Nov 22, 2024
8dd3cc4
Feedback: update filterHandler
raejohanek Nov 22, 2024
d03c0bf
Feedback: add validation
raejohanek Nov 22, 2024
e1383f1
Merge branch 'develop' into rj/dt-777-filter-participant-count
raejohanek Nov 22, 2024
d0556f8
try to fix test
raejohanek Nov 22, 2024
c033344
try to fix test
raejohanek Nov 22, 2024
4cc1ac8
try to fix test
raejohanek Nov 22, 2024
44e9f1c
try to fix test
raejohanek Nov 22, 2024
2417d78
fix tests?
raejohanek Nov 25, 2024
af1a501
fails locally, but just curious if it will pass on the server
raejohanek Nov 25, 2024
24ca3b1
revert last change
raejohanek Nov 25, 2024
c4ed6ee
reduce URLs and rename parameter
raejohanek Nov 25, 2024
2d3b3ab
remove debounce just to see
raejohanek Nov 25, 2024
851879e
add cy.clock and revert remove debounce
raejohanek Nov 25, 2024
455abe3
add cy.clock to existing tests!!
raejohanek Nov 26, 2024
ff293e5
pr feedback: simplify anyFiltersSelected and rename response alias in…
raejohanek Nov 26, 2024
79eaba9
pr feedback: update calls in test
raejohanek Dec 2, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 14 additions & 18 deletions cypress/component/DataSearch/dataset_search_filters.spec.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,26 +2,22 @@

import { mount } from 'cypress/react';
import React from 'react';
import {Storage} from '../../../src/libs/storage';
import DatasetFilterList from '../../../src/components/data_search/DatasetFilterList';

const duosUser = {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unused

isSigningOfficial: false,
};

describe('Data Library Filters', () => {
// Intercept configuration calls
beforeEach(() => {
cy.intercept({
method: 'GET',
url: '/config.json',
hostname: 'localhost',
}, { 'env': 'ci' });
});
// Intercept configuration calls
beforeEach(() => {
cy.initApplicationConfig();
});

it('Renders the data library filters', () => {
const props = { datasets: [], filters: [], filterHandler: () => {}, isFiltered: () => {}};
mount(<DatasetFilterList {...props} />);
cy.get('div').should('contain', 'Filters');
});
it('Renders the data library filters', () => {
const props = { datasets: [], filterHandler: () => {}, isFiltered: () => {}};
mount(<DatasetFilterList {...props} />);
cy.get('div').should('contain', 'Filters');
cy.get('div').should('contain', 'Access Type');
cy.get('div').should('contain', 'Data Use');
cy.get('div').should('contain', 'Data Access Committee');
cy.get('div').should('contain', 'Data Type');
cy.get('div').should('contain', 'Participant Count');
});
});
30 changes: 29 additions & 1 deletion cypress/component/DataSearch/dataset_search_table.spec.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,9 @@ const datasets = [
datasetId: 123456,
datasetIdentifier: `DUOS-123456`,
datasetName: 'Some Dataset 1',
participantCount: 100,
study: {
studyName: 'Some Study 1',
studyId: 1,
dataCustodianEmail: ['Some Data Custodian Email 1'],
}
Expand All @@ -23,8 +25,9 @@ const props = {

describe('Dataset Search Table tests', () => {

describe('Data library with three datasets', () => {
describe('Data library with one dataset footer tests', () => {
beforeEach(() => {
cy.initApplicationConfig();
cy.stub(TerraDataRepo, 'listSnapshotsByDatasetIds').returns({});
mount(<DatasetSearchTable {...props} />);
});
Expand All @@ -38,6 +41,31 @@ describe('Dataset Search Table tests', () => {
cy.get('#header-checkbox').click();
cy.contains('1 dataset selected from 1 study');
});
});

describe('Data library filter by participant count tests', () => {
beforeEach(() => {
cy.initApplicationConfig();
cy.stub(TerraDataRepo, 'listSnapshotsByDatasetIds').returns({});
});

it('When a participant count filter is applied the query is updated', () => {
var filtered = false;
function handler(request) {
if (JSON.stringify(request.body).includes('{"range":{"participantCount":{"gte":null,"lte":"50"}}}')) {
filtered = true;
}
request.reply({statusCode: 200, body:[]});
}

cy.intercept(
{method: 'POST', url: '**/api/dataset/search/index'}, handler).as('searchIndex');
mount(<DatasetSearchTable {...props} />);
cy.get('#participantCountMax-range-input').clear().type('50');
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cypress recommends not chaining calls after clear. It's a little more verbose, but safer to:

      const range = cy.get('#participantCountMax-range-input');
      range.clear();
      range.type('50');

cy.wait(1000).then(() => {
rjohanek marked this conversation as resolved.
Show resolved Hide resolved
expect(filtered).to.be.true;
});
});

});
});
42 changes: 36 additions & 6 deletions src/components/data_search/DatasetFilterList.jsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ import ListItemButton from '@mui/material/ListItemButton';
import ListItemIcon from '@mui/material/ListItemIcon';
import ListItemText from '@mui/material/ListItemText';
import Divider from '@mui/material/Divider';
import { Button, Typography } from '@mui/material';
import { Button, TextField, Typography } from '@mui/material';
import { Checkbox } from '@mui/material';
import {flatten, uniq, compact, orderBy} from 'lodash';
import {getAccessManagementSummary} from '../../types/model';
Expand All @@ -21,15 +21,15 @@ export const FilterItemHeader = (props) => {
};

export const FilterItemList = (props) => {
const { category, datasets, filter, filterHandler, isFiltered, filterNameFn, filterDisplayFn } = props;
const { category, filter, filterHandler, isFiltered, filterNameFn, filterDisplayFn } = props;
return (
<List sx={{ margin: '-0.5em -0.5em' }}>
{
filter.map((filter) => {
const filterName = filterNameFn(filter);
return (
<ListItem disablePadding key={filter}>
<ListItemButton sx={{ padding: '0' }} onClick={(event) => filterHandler(event, datasets, category, filter)}>
<ListItemButton sx={{ padding: '0' }} onClick={() => filterHandler(category, filter)}>
<ListItemIcon>
<Checkbox checked={isFiltered(filter, category)} />
</ListItemIcon>
Expand All @@ -45,13 +45,34 @@ export const FilterItemList = (props) => {
);
};

export const FilterItemRange = (props) => {
const { min, max, minCategory, maxCategory, filterHandler } = props;
return (
<Box key={minCategory + '-' + maxCategory} sx={{ display: 'flex', flexDirection: 'row', alignItems: 'center' }}>
<TextField id={minCategory + '-range-input'} size='small' margin='dense' variant='outlined' defaultValue={min}
helperText={'minimum'}
FormHelperTextProps={{style: { transform: 'scale(1.5)' }}}
onChange={(event) => filterHandler(minCategory, event.target.value === '' ? min : event.target.value)}/>
<Box padding={'0rem 1rem 1rem'}> - </Box>
<TextField id={maxCategory + '-range-input'} size='small' margin='dense' variant='outlined' defaultValue={max}
helperText={'maximum'}
FormHelperTextProps={{style: {transform: 'scale(1.5)'}}}
onChange={(event) => filterHandler(maxCategory, event.target.value === '' ? max : event.target.value)}
/>
</Box>
);
};

export const DatasetFilterList = (props) => {
const { datasets, filters, filterHandler, isFiltered, onClear } = props;
const { datasets, filterHandler, isFiltered, onClear } = props;

const accessManagementFilters = uniq(compact(datasets.map((dataset) => dataset.accessManagement)));
const dataUseFilters = uniq(compact(flatten(datasets.map((dataset) => dataset.dataUse?.primary))).map((dataUse) => dataUse.code));
const dataTypeFilters = uniq(flatten(datasets.map((dataset) => dataset.study.dataTypes)));
const dacFilters = orderBy(uniq(compact(datasets.map((dataset) => dataset.dac?.dacName))), (dac) => dac.toLowerCase(), 'asc');
// some participantCounts are undefined, so filter them out before calculating min and max
const participantCountMax = Math.max(...datasets.filter((dataset) => dataset.participantCount).map((dataset) => dataset.participantCount));
rjohanek marked this conversation as resolved.
Show resolved Hide resolved
const participantCountMin = Math.min(...datasets.filter((dataset) => dataset.participantCount).map((dataset) => dataset.participantCount));

return (
<Box sx={{ bgcolor: 'background.paper' }}>
Expand Down Expand Up @@ -101,15 +122,24 @@ export const DatasetFilterList = (props) => {
isFiltered={isFiltered}
filterNameFn={(filter) => filter}
/>
<FilterItemHeader title="Data Type" />
<FilterItemHeader title='Data Type' />
<FilterItemList
category="dataType"
category='dataType'
datasets={datasets}
filter={dataTypeFilters}
filterHandler={filterHandler}
isFiltered={isFiltered}
filterNameFn={(filter) => filter}
/>
<FilterItemHeader title='Participant Count' />
<FilterItemRange
min={participantCountMin}
max={participantCountMax}
minCategory='participantCountMin'
rjohanek marked this conversation as resolved.
Show resolved Hide resolved
maxCategory='participantCountMax'
datasets={datasets}
filterHandler={filterHandler}
/>
</Box>
);
};
Expand Down
46 changes: 37 additions & 9 deletions src/components/data_search/DatasetSearchTable.jsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import useOnMount from '@mui/utils/useOnMount';
import * as React from 'react';
import { Box, Button } from '@mui/material';
import { useEffect, useRef, useState } from 'react';
import { isEmpty } from 'lodash';
import { isArray, isEmpty } from 'lodash';
import { TerraDataRepo } from '../../libs/ajax/TerraDataRepo';
import { DatasetSearchTableDisplay } from './DatasetSearchTableDisplay';
import { datasetSearchTableTabs } from './DatasetSearchTableConstants';
Expand Down Expand Up @@ -37,7 +37,9 @@ const defaultFilters = {
accessManagement: [],
dataUse: [],
dataType: [],
dac: []
dac: [],
participantCountMin: null,
participantCountMax: null,
};

export const DatasetSearchTable = (props) => {
Expand All @@ -49,8 +51,21 @@ export const DatasetSearchTable = (props) => {
const [selectedTable, setSelectedTable] = useState(datasetSearchTableTabs.study);
const [searchTerm, setSearchTerm] = useState('');

const isFiltered = (filter, category) => (filters[category]).indexOf(filter) > -1;
const numSelectedFilters = (filters) => Object.values(filters).reduce((sum, array) => sum + array.length, 0);
const isFilteredArray = (filter, category) => (filters[category]).indexOf(filter) > -1;

const numSelectedFilters = (filters) => {
rjohanek marked this conversation as resolved.
Show resolved Hide resolved
var sum = 0;
for (const category in filters) {
if (isArray(filters[category])) {
sum += filters[category].length;
} else {
if (filters[category]) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would combine an if after an else here

Suggested change
} else {
if (filters[category]) {
} else if (filters[category]) {

sum += 1;
}
}
}
return sum;
};

const getExportableDatasets = async (datasets) => {
// Note the dataset identifier is in each sub-table row.
Expand Down Expand Up @@ -155,6 +170,15 @@ export const DatasetSearchTable = (props) => {
}
});

filterTerms.push({
'range': {
'participantCount': {
'gte': filters.participantCountMin,
'lte': filters.participantCountMax,
rjohanek marked this conversation as resolved.
Show resolved Hide resolved
}
}
});

if (filterTerms.length > 0) {
filterQuery = [
{
Expand All @@ -179,12 +203,16 @@ export const DatasetSearchTable = (props) => {
};
};

const filterHandler = (event, data, category, filter) => {
const filterHandler = (category, filter) => {
var newFilters = _.clone(filters);
if (!isFiltered(filter, category) && filter !== '') {
newFilters[category] = filters[category].concat(filter);
if (isArray(newFilters[category])) {
if (!isFilteredArray(filter, category) && filter !== '') {
newFilters[category] = filters[category].concat(filter);
} else {
newFilters[category] = filters[category].filter((f) => f !== filter);
}
} else {
newFilters[category] = filters[category].filter((f) => f !== filter);
newFilters[category] = filter;
}
setFilters(newFilters);
rjohanek marked this conversation as resolved.
Show resolved Hide resolved
};
Expand Down Expand Up @@ -280,7 +308,7 @@ export const DatasetSearchTable = (props) => {
</Box>
<Box sx={{display: 'flex', flexDirection: 'row', paddingTop: '2em'}}>
<Box sx={{width: '14%', padding: '0 1em'}}>
<DatasetFilterList datasets={datasets} filters={filters} filterHandler={filterHandler} isFiltered={isFiltered} onClear={() => setFilters(defaultFilters)}/>
<DatasetFilterList datasets={datasets} filterHandler={filterHandler} isFiltered={isFilteredArray} onClear={() => setFilters(defaultFilters)}/>
</Box>
<Box sx={{width: '85%', padding: '0 1em'}}>
{(() => {
Expand Down