Explicit warning / error if patients.which_exist_in_file tries to read in an empty dataset

[wjchulme]

I tried to run a study definition but hit this error:

feather does not support serializing <class 'pandas.core.indexes.base.Index'> for the index; you can .reset_index() to make the index into column(s)`

This doesn't happen when running locally, only on the server.

• job request
• study definition

I assume this is something that needs to be fixed in the cohort-extractor code rather than something odd going on with my code, but I don't really know how to probe further.

[inglesp]

This is consistent with an attempt to serialize an empty dataframe, and I can reproduce it by removing all non-header rows from this test data.

Are you sure that your study definition is returning a population?

[evansd]

@inglesp's diagnosis is correct:

The population is empty because output/match/cumulative_matchedcontrols1.csv.gz is empty.

I haven't debugged further than that.

[wjchulme]

Thanks, both. If i'd known that error was hit when saving an empty dataset I wouldn't have raised this issue. I'll spare you the details but I've found the problem -- a bug arising from trying to retain within-patient consistency in dummy data across multiple extracts.

Might be worth an explicit error if patients.which_exist_in_file is reading in an empty dataset?