Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add schema collection to Postgres integration #15484

Merged
merged 133 commits into from
Aug 16, 2023
Merged

Add schema collection to Postgres integration #15484

merged 133 commits into from
Aug 16, 2023

Conversation

edengorevoy
Copy link
Contributor

@edengorevoy edengorevoy commented Aug 4, 2023 •
edited
Loading

What does this PR do?

This PR enables schema collection in the agent, per this RFC.

Motivation

We want to be able to see schemas per host in the UI, so collection in the agent is the first step to implementing this.

Additional Notes

Review checklist (to be filled by reviewers)

  • Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
  • PR title must be written as a CHANGELOG entry (see why)
  • Files changes must correspond to the primary purpose of the PR as described in the title (small unrelated changes should have their own PR)
  • PR must have changelog/ and integration/ labels attached
  • If the PR doesn't need to be tested during QA, please add a qa/skip-qa label.

@edengorevoy
first commit
829e905
@edengorevoy
All schema collection except ordering by metrics
872a5ac
@edengorevoy
Ordering tables by relation metrics
feb06dc
@edengorevoy
toast table changes
5c62cfe
@edengorevoy
Adding some testing, and updating the payload
6c97b86
@edengorevoy
Add a debug log
3a1a8cd
@edengorevoy
Updating last_schemas_query at correct time
9282551
@edengorevoy
Actually return the metadata from _collect_schema_info
4f3bdff
@edengorevoy
Convert db id from int to str
5c61488
@edengorevoy
Correct json objects for indexes, foreign keys
d052c9e
@edengorevoy
Correct json objects for indexes, foreign keys
5c560bc
@edengorevoy
Updating queries and toast table getting
1254049
@edengorevoy
Config update
e1e6905
@edengorevoy
Updating to include OIDs in return, and don't calculate fk for tables…
ca3ee13 
… that don't have them
@edengorevoy
Update some comments
1a5a473
@edengorevoy
Make test mor specific
d380f13
@edengorevoy
Update config to limit columns
8336f17
@edengorevoy
Formatting
7fd65a0
@edengorevoy
Remove excess warnings and prints
9a4c7bb
@edengorevoy
Fix schema id propagation
4e5266b
@edengorevoy
Clean up caching
4df28a6
@edengorevoy
Formatting, updating to psycopg3
c7b9394
@edengorevoy
Conditionals for postgres version 9, no partitions
864f2ba
@edengorevoy
Update table size test
74eb63e
@github-actions
Copy link

github-actions bot commented Aug 4, 2023

The validations job has failed; please review the Files changed tab for possible suggestions to resolve.

@edengorevoy edengorevoy marked this pull request as ready for review August 4, 2023 19:10
@edengorevoy edengorevoy requested a review from a team as a code owner August 4, 2023 19:10
@edengorevoy edengorevoy force-pushed the collect-postgres-schemas branch 3 times, most recently from 4ddeea0 to 5f79935 Compare August 15, 2023 18:37
@alexandre-normand
Copy link
Contributor

Looks the merge of my PR is causing conflicts for you. Sorry about that. But this is looking good otherwise.

jmeunier28
jmeunier28 previously approved these changes Aug 16, 2023
View reviewed changes
Copy link
Contributor

@jmeunier28 jmeunier28 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🥳

jmeunier28
jmeunier28 previously approved these changes Aug 16, 2023
View reviewed changes
@jmeunier28 jmeunier28 merged commit 954042f into master Aug 16, 2023
27 checks passed
@jmeunier28 jmeunier28 deleted the collect-postgres-schemas branch August 16, 2023 23:21
jmeunier28 pushed a commit that referenced this pull request Sep 19, 2023
@edengorevoy @jmeunier28
Add schema collection to Postgres integration (#15484)
300ed3c
@jmeunier28 jmeunier28 mentioned this pull request Sep 19, 2023
Merged
3 tasks
jmeunier28 added a commit that referenced this pull request Sep 19, 2023
@jmeunier28
Add schema collection to Postgres integration (#15484) (#15866)
d25a65e
vivek-datadog pushed a commit that referenced this pull request Sep 19, 2023
@jmeunier28 @vivek-datadog
Add schema collection to Postgres integration (#15484) (#15866)
469523d
@FlorentClarret FlorentClarret mentioned this pull request Oct 26, 2023
Merged
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@alexandre-normand alexandre-normand alexandre-normand approved these changes

@jmeunier28 jmeunier28 jmeunier28 left review comments

@urseberry urseberry urseberry left review comments

Assignees
No one assigned
Labels
documentation integration/postgres integration/yarn
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@edengorevoy @alexandre-normand @jmeunier28 @urseberry