BCDA-8350: Add workflow to generate database docs

[kyeah]

🎫 Ticket

https://jira.cms.gov/browse/BCDA-8350

🛠 Changes

• Adds database documentation in the repo using https://github.com/k1LoW/tbls
• Adds github workflow to automatically update PRs that touch the migrations folders

ℹ️ Context

As part of BCDA-8350, I was interested in an easier way to keep the documentation up to date. Although ERDs can be generated manually by tooling like pgAdmin, it would be preferable for devs to have the diagrams generated automatically in a central location.

We can still pull the main ERDs into confluence, but instead of making them ourselves we can download them directly from the PR after merging.

🧪 Validation

See dbdocs in this PR.

[carlpartridge]

Do suppresions have one suppression_files (file_id?)?

The foreign_key connection is a bit wonky on the schema.svg but overall this seems great.

My understanding is that this DB is shared with SSAS which would include tables like systems and groups. Do you think it is problematic (or fortunate) that those are missing here?

[carlpartridge]

These github actions dont leave any kind of artifacts anywhere I assume?

[kyeah]

oh hm! I don't think so, I copied this from the existing ci-workflow.yml. We're good to keep decrypted secrets on the public GA machines, right @CMSgov/ab2d-bcda-dpc-platform?

[gsf]

Better not to leave them lying around if we can avoid it. There was a vulnerability found recently in GitHub Actions artifacts. That doesn't apply directly here, but does point toward the potential for leaks.

[carlpartridge]

Seeing a failure on generate-docs, I think you need to add a --force to it to make sure it is overwriting the existing docs (https://github.com/k1LoW/tbls?tab=readme-ov-file#re-generating-database-documentation)?

[kyeah]

Do suppressions have one suppression_files (file_id?)?

yup, each suppression is from a single file! subsequent changes for a single MBI have different records with different file_ids, so we grab the one with the latest timestamp.

The foreign_key connection is a bit wonky on the schema.svg but overall this seems great.

yeah, i didn't see any configuration to either hide the text or make it prettier.

My understanding is that this DB is shared with SSAS which would include tables like systems and groups. Do you think it is problematic (or fortunate) that those are missing here?

oops yes! I forgot to run the SSAS table migrations (those live in the bcda-ssas-app repo.) Hmm that also makes autogeneration difficult — let me look at the best way to do that (I think maybe add a separate workflow to generate docs just for the SSAS app within the other repo?)

[carlpartridge]

My understanding is that this DB is shared with SSAS which would include tables like systems and groups. Do you think it is problematic (or fortunate) that those are missing here?

oops yes! I forgot to run the SSAS table migrations (those live in the bcda-ssas-app repo.) Hmm that also makes autogeneration difficult — let me look at the best way to do that (I think maybe add a separate workflow to generate docs just for the SSAS app within the other repo?)

If the BCDA doesnt "know" about systems and groups, etc then to some degree it makes sense to leave them out of the bcda-app repo documentation. We can pretty easily recreate these docs in bcda-ssas-app, which would have the other side of things (unless ssas references bcda-app tables?). Still a bit awkward to have one DB docs spread out across two repos.
Overall it feels a bit messy to have multiple apps reaching into the same DB, especially migrating structure and data.

[kyeah]

Yeah, I think lets keep the docs separate and we can mentally treat them as two separate databases since that's how they operate if i recall correctly. The fact that they actually live in the same database is more of an odd implementation detail 🙃

[carlpartridge]

Do we need to rm .vault_password and maybe shared_files/encrypted as well?

[carlpartridge]

LGTM but you might want to get Sean's sign off as well.