Error in Cohort Characterization tab

[bdemeulder]

In the shiny app cohort characterisation, there seems to be an error in parsing the results.

Further, when looking only at the demographics domain, it seems the rows are replicated many times in the table (e.g. more than 170 occurences of gender=male)

Compare cohorts characteristics seems fine.

[MaximMoinat]

That indeed looks like a join/merge gone wrong, but I can't reproduce this with the data on my side. What premerge.R data file are you using? @bdemeulder

[bdemeulder]

@MaximMoinat I'm using the one I've just uploaded to teams (task #5/Processed results files/results)

[MaximMoinat]

I found the issue, the function getCovariateDataSubset returns data for all cohorts while it should filter for the selected cohort. So the duplication that we are seeing are the covariates for all cohorts. We are not seeing this in the cohort comparison tab as this filters in a different way.

PioneerWatchfulWaiting/inst/shiny/PioneerWatchfulWaitingExplorer/server.R

Lines 60 to 66 in 21c54fe

	getCovariateDataSubset <- function(cohortId, databaseList, comparatorCohortId = NULL) {
	if (usingDbStorage()) {
	return(getCovariateValue(connPool, cohortId = cohortId, databaseList = databaseList, comparatorCohortId = comparatorCohortId))
	} else {
	return(covariateValue[covariateValue$cohortId %in% c(cohortId, comparatorCohortId) & covariateValue$databaseId %in% databaseList, ])
	}
	}

Debugging further to find a solution.

[MaximMoinat]

PR #95 fixes an issue on the shiny side. However, there is another issue with duplicate covariateIds. See following screenshot, where e.g. id 462874664123 is in there twice, with a different name. They should have different windowId's and therefore a different last digit.

It seems like databases have conflicting sets of covariateIds. Could it be one or more have run a different version of the study package?

[bdemeulder]

Checked in the different datasets, there are indeed two sets of results with different covariate IDs.
When generating a new premerge.Rdata with only CPRD, MarketScan, OPTUM (the three from Bayer) + MAITT and TMC, there are no issues anymore.

Those seem to agree on the covariate table.

So indeed, some datasets might have run different version of the package.

[bdemeulder]

@MaximMoinat, @keesvanbochove: how can we check what the proper covariate table should be in the current version, so we can pinpoint which datasets have run the correct version and which should run it again?

[ablack3]

The version of the results viewer on https://data.ohdsi.org/PioneerWatchfulWaiting/ still has this issue. The download button on the characterization tab fails as well.

[keesvanbochove]

The version of the results viewer on https://data.ohdsi.org/PioneerWatchfulWaiting/ still has this issue. The download button on the characterization tab fails as well.

@bdemeulder could you upload the new merged set to the OHDSI server so this could be fixed online as well?

[bdemeulder]

@keesvanbochove , I've updated the merged set and created pull request #134 on shiny deploy to trigger the update. Could you approve it?

[ahijazy]

Facing the same issue, could you please clarify how was the error finally solved?