Skip to content

Datasetmanager tests race condition #559

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
rettigl opened this issue Feb 7, 2025 · 2 comments · May be fixed by #562
Open

Datasetmanager tests race condition #559

rettigl opened this issue Feb 7, 2025 · 2 comments · May be fixed by #562
Assignees

Comments

@rettigl
Copy link
Member

rettigl commented Feb 7, 2025

DatasetManager tests fail some times if executed concurrently. This is because all tests access the same global dataset configuration. Also, running the tests leaves a "test" folder behind. This should be avoided by properly mocking paths, and using tmp_path.

@zain-sohail zain-sohail linked a pull request Feb 27, 2025 that will close this issue
@zain-sohail
Copy link
Member

@Aserhisham Potentially a simpler first issue to tackle compared to writing lab loader tests. I tried to fix it in the PR #562 but the race conditions are still there. If you prefer, you can ignore my changes and work from main branch, or build on top of the PR.

@rettigl
Copy link
Member Author

rettigl commented May 11, 2025

I recently learned there is a convenient way to avoid this by using load groups. Using this approach you can place all tests that use similar resources into the same worker thread. See e.g. here: FAIRmat-NFDI/pynxtools#626

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants