Should we split image builds and terraform modules into their own repositories or continue to manage them in Data Platform Monorepo?

[julialawrence]

Context:

We are currently pursuing an unsustainable strategy of managing some image builds in Data Platform monorepo and some in their own repositories.

We need to align around one solution for this so we can all implement it, and we can alter how we manage OIDC roles and other IAM permissions to support one approach or another.

Both approaches have their benefits:

Data Platform:

• All code in one place
• Consistent workflows
• Improved visibility on what modules/images we already have which encourages reuse

Separate repos:

• Versioning is a lot more intuitive and easily externally referenced
• Commit histories fully applicable to the image or module code making auditing changes a lot easier
• Approach mostly employed elsewhere around Justice Digital
• Simplified unit testing

Assumptions:

• Whatever the outcome, the repo(s) where these components are managed is/are fully managed and configured in terraform so as to avoid the issue of unending unearthing of random repositories.
• Locally used/declared modules are out of scope.

[MatMoore]

My vote is for splitting, just because it is really common for tooling around github to assume 1 artefact per repo, so it feels like if you go with a monorepo you have to do extra work to make the tools work for you, rather than just relying on defaults.

An example is the dependabot changelog helper you mentioned in another discussion. If we were using 1 artefact per repo, we could just enable it and rely on the defaults, but since we have multiple builds per repo we need to figure out how to configure it to trigger with the correct path depending on what's changed in the PR.

[julialawrence]

Implementation issue raised, closing.

#2502