Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cleanup use of federation metadata in client / avoid race conditions #744

Closed
bbockelm opened this issue Feb 2, 2024 · 0 comments · Fixed by #878
Closed

Cleanup use of federation metadata in client / avoid race conditions #744

bbockelm opened this issue Feb 2, 2024 · 0 comments · Fixed by #878
Assignees
Labels
bug Something isn't working client Issue affecting the OSDF client critical High priority for next release
Milestone

Comments

@bbockelm
Copy link
Collaborator

bbockelm commented Feb 2, 2024

When cleaning up #721, it became obvious that the client code structure is wrong in its handling of metadata.

The client uses the global discovery metadata. However, the "correct" director, for example, may change URL-by-URL -- and if we are downloading multiple URLs in parallel, there's going to be race conditions setting/unsetting the global director location for each URL we parse.

Instead of reaching into the global we should create a URL-like structure that holds the metadata along with the object name. We should also keep a ttlcache of metadata for different federations to prevent frequent lookups if we are parsing many URLs within the process.

@bbockelm bbockelm added bug Something isn't working client Issue affecting the OSDF client critical High priority for next release labels Feb 2, 2024
@bbockelm bbockelm added this to the v7.6.0 milestone Feb 2, 2024
@joereuss12 joereuss12 linked a pull request Feb 22, 2024 that will close this issue
@joereuss12 joereuss12 linked a pull request Mar 1, 2024 that will close this issue
@bbockelm bbockelm modified the milestones: v7.6.0, v7.7.0 Mar 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working client Issue affecting the OSDF client critical High priority for next release
Projects
None yet
2 participants