Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes 4740: Change base query to use inner joins #13

Merged
merged 1 commit into from
Oct 2, 2024

Conversation

jlsherrill
Copy link
Member

@jlsherrill jlsherrill commented Sep 25, 2024

I saw improvements with package queries by around 2x. Errata query times were very similar to before.

To test:

Load a bunch of repos and sync them (make repos-import && go run cmd/external-repos/main.go nightly-jobs). You might add some large fedora repos too.

in content-sources-backend go.mod add this under go 1.23:

replace github.com/content-services/tang => /home/jlsherri/git/tang

(replace with path to your checkout).

go get ./...

(Note that when switching branches, or modifying code in tang, you must rm release/content-sources before you make run to pick up the changes

  • Confirm that listing rpms and advisories for templates works
  • compare the counts before and after this change
  • To see a time difference, you can enable query logging in postgres:
psql --host localhost -U pulp
ALTER SYSTEM SET log_min_duration_statement = 25;
select pg_reload_conf();

Now after fetching some set of content, you can run podman logs --tail 1000 cs_postgres_1 and see the time it takes:

LOG:  duration: 208.538 ms

You can switch back and forth between main and this branch to see the changes. You may want to compare the times (should be about half).

@jlsherrill jlsherrill marked this pull request as ready for review September 25, 2024 18:28
@rverdile rverdile self-assigned this Sep 26, 2024
@rverdile
Copy link
Collaborator

rverdile commented Oct 1, 2024

also seeing ~2x speedup!

@jlsherrill jlsherrill merged commit f7e88f5 into content-services:main Oct 2, 2024
2 checks passed
@jlsherrill jlsherrill deleted the query_speedup branch October 2, 2024 22:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants