-
Notifications
You must be signed in to change notification settings - Fork 75
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Pet Pipeline: Non-Selective Reprocessing of Eligible Bids Entries #1079
Comments
Hi @souravraha Thanks for pointing this out. I'm not sure to fully understand what you mean though. Could you share the commands that you executed ? |
@NicolasGensollen
I noticed that the PET pipeline processes each BIDS subject sequentially, despite having previously executed the pipeline. In contrast, the T1 pipeline issues a warning indicating that each BIDS subject has already been processed and promptly skips them, resulting in a significantly shorter re-execution time, typically completing within a few seconds. |
@souravraha I think you're right. There is some logic implemented in the clinica/clinica/pipelines/t1_linear/anat_linear_pipeline.py Lines 136 to 153 in 2939b05
This is based on the implementation of this method: clinica/clinica/pipelines/t1_linear/anat_linear_pipeline.py Lines 28 to 41 in 2939b05
Which has an abstract definition in the engine, but is not implemented by all pipelines (for example PET pipelines do not implement this). What's even more strange is that some pipelines (like I think we should definitely fix this and offer a similar user experience for all pipelines. I'll add this to my todo list 😅 |
While discussing this matter, I'd like to revisit a previous issue, #1060, which you helped resolve. After incorporating your enhancements and executing the converter on the existing bids directory, I encountered the same errors mentioned in #1060. To resolve this, I delved deeper and identified files with troublesome suffixes ("ADC", "real") within the bids directory. These files stemmed from an earlier problematic version of the converter. Upon removing these older files, I was able to successfully execute the converter with your enhancements. It appears that any improvements implemented may not be effective until the problematic files are removed from the disk. This issue could potentially be mitigated if the logic initially detected such problematic files. I wanted to bring this to your attention for your awareness. |
This issue is considered stale because it has not received further activity for the last 14 days. You may remove the |
This issue is considered stale because it has not received further activity for the last 14 days. You may remove the |
Describe the bug
The pet pipeline is not intelligently skipping already processed bids entries, unlike the t1 pipeline. This results in the pet pipeline reprocessing all eligible bids entries.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
The pet pipeline should intelligently skip already processed bids entries, similar to the behavior of the t1 pipeline.
Screenshots
N/A
Desktop:
Additional context
This issue affects the efficiency and performance of the pet pipeline. Implementing intelligent skipping of already processed bids entries will significantly improve its performance and reduce redundant processing.
The text was updated successfully, but these errors were encountered: