Several fixes after running a full dataset #24
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
nictru
reviewed
Dec 19, 2024
conf/modules.config
Outdated
Comment on lines
128
to
130
| withName: "RUN_DYNAMITE" { | ||
| errorStrategy = "ignore" | ||
| } |
There was a problem hiding this comment.
In what situations is this relevant? By setting the errorStrategy to ignore, we also prevent the pipeline from trying again if it fails due to too little RAM etc
There was a problem hiding this comment.
Dynamite fails with exitStatus 139 when running with too little data, so I adapted the errorStrategy accordingly.
Comment on lines
-194
to
+196
| rndselect=sample(x=nrow(M),size=as.numeric(argsL$testsize)*nrow(M)) | ||
| # Test on a single example if dataset size is too small | ||
| rndselect=sample(x=nrow(M),size=ifelse(as.numeric(argsL$testsize)*nrow(M) < 1, 1, as.numeric(argsL$testsize)*nrow(M))) |
There was a problem hiding this comment.
This file is a copy from here and I would like to keep it identical if possible
There was a problem hiding this comment.
We could open a PR to their repository, however, the tool does not appear to be actively maintained. The last open PR is from 2020 and still unanswered.
The changes improve the tool's robustness when handling small dataset sizes, making it essential for a reliable pipeline and also for the run on our lactation data.
| df_lengths = df_lengths.loc[df_counts.index] | ||
| df_lengths = df_lengths.loc[df_lengths.index.isin(df_counts.index)] |
There was a problem hiding this comment.
Did you check if it can happen that the dataframes have different orders?
If we cannot be entirely sure of this, it might be better to build the intersection of both indices and then subset both to the intersection
There was a problem hiding this comment.
I switched to the index intersection you proposed. Seems not necessary for our use case, since both data frames have the same order, but it's definitely more robust this way.
| @@ -14,38 +15,34 @@ def format_yaml_like(data: dict, indent: int = 0) -> str: | |||
| """ | |||
| yaml_str = "" | |||
| for key, value in data.items(): | |||
| spaces = " " * indent | |||
| spaces = " " * indent | |||
There was a problem hiding this comment.
I guess this was not on purpose
There was a problem hiding this comment.
I think four spaces are the default for nf-core modules.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes:
binarizeBamsandLearnModelDYNAMITE:PREPROCESSTF_TG_SCORE