Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

oslcquery speed up large --percontribution querying by parallelising query with processing of results from previous query #21

Open
barny opened this issue Sep 28, 2022 · 0 comments
Labels
enhancement New feature or request

Comments

@barny
Copy link
Member

barny commented Sep 28, 2022

One example doing counting of artifacts/module in a very large GC project with 1253 contributions containing 25886 modules and total of 6722401 bindings was spending about 50% of the time processing results, sequentially with the queries; if the processing can be done in parallel with the next query that's a significant speedup. Probably doesn't need anything more fancy/complicated than processing in parallel with the next query, but I suppose some more speedup might be possible with a controlled number of queries in parallel, at the cost of server load.

-q rdm_types:ArtifactFormat=jazz_rm:Module -s oslc_rm:uses,dcterms:title,dcterms:identifier

@barny barny added the enhancement New feature or request label Sep 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant