Approval of breaking schema changes through Hive-CLI

[jenskm]

Hello,

We aim to refine our CI pipeline by allowing the pull request reviewer to approve breaking schema changes directly within the pull request. Currently, a breaking schema change results in a failed Hive schema check, necessitating manual approval in the Hive App. Once approved, we manually rerun the CI pipeline to achieve a 'green' status, signifying an error-free schema check.

Our proposal is to make Hive-CLI able to handle the approval of breaking schema changes. We could then trigger this from our pipeline, e.g. in a similar way that the merge request itself is approved. This would streamline our workflow by eliminating the need to switch between platforms and manually trigger the pipeline rerun.

We welcome your thoughts on this proposal.

Best Regards, Jens Kilde Mjelva

[PlayAnyData]

Hey,
we also thought about this, a while ago. We dropped this, because we want the "extra mile" to be done here, as devs should explicitly check the (full) schema check.

However, how do you imagine this to work? Like, what would a process look like here?

[jenskm]

Hello,

Thank you for your feedback!

Here’s a rough sketch of how I envision this process:

1. A developer or schema designer initiates a schema change with breaking changes via a pull request.
2. As with our current system, the Hive schema check fails due to the proposed change.
3. Instead of manually approving the change through the Hive App, the pull request reviewer triggers a pipeline job. This job issues a Hive-CLI command (hive schema:approve) within the pull request itself to approve the schema change, optionally along with a justification for the necessary breaking change.
4. Following approval, the CI pipeline is rerun automatically. The schema check returns ok and if the rest of the code passes review, the pipeline will return to a ‘green’ status.

We're also contemplating a simpler alternative that may better suit our immediate needs. We could require just a single approval from the pull request reviewer, wherein the approval of the pull request itself would also serve as the approval for any breaking schema changes. This would eliminate the need for separate approval of schema changes in Hive. In this scenario, we run the hive schema:publish command upon merging the pull request, without having the changes explicitly approved in Hive.

It's important to mention that our schema contains "experimental features." Consumers are aware these may break, leading to many anticipated and desired breaking changes. Integrating the approval of these changes into the pull request process can make handling them more efficient.

Your thoughts and suggestions are very welcome.