discuss: Automate OpenDAL Release

[Xuanwo]

Hello everyone, I'm starting this thread to discuss whether it's possible to fully automate the OpenDAL release process.

Goal

• The Release Manager no longer needs to handle tedious tasks. CI will manage artifact signing and SVN uploads.
• Committers no longer need to perform manual verifications. CI will handle all verification steps within the release workflow.
• The community no longer needs to wait 72 hours. OpenDAL PMC members should review the CI verification source code and logs to cast their votes. Once three votes are collected, the Release Manager can push the official tag.

For each release, the following steps are required:

• The Release Manager should push SOME RC tags and ONE release tag, with no additional actions.
• The Committer should review ONE verification code and logs to cast ONE vote, with no further actions.

The verification log will be saved and uploaded to the GitHub release as part of the release for future reference.

Implement

Most of the work will take place in CI, with details not elaborated here. Most of it is simply implementation.

To make this possible, we will need a dedicated SVN account to carry out the upload process on behalf of the OpenDAL community.

We will not depends on GPG key signing. Instead, we will integrate with sigstore and mechanism like Github Artifact Attestations to make sure the artifacts not changed.

• Users are assured that these artifacts are produced by our workflow.
• Users can visit our provenance through a public registry.
• Users can verify our releases using tools like Sigstore or gh.

All of this ensures we provide better guarantees than a simple GPG signature.

Context

Inspired by my post: What did ASF do wrong?.

The OpenDAL PMC is the first PMC to undertake such experimentation.

[xushiyan]

@Xuanwo thanks for bringing up this important topic - I'm fully aligned with the goal here, and I want to employ the same setup for https://github.com/apache/hudi-rs

Currently in hudi-rs, we have automated the package artifacts publishing via CI and by pushing release-* tags, but not the code-signing and src release part using gpg. This part is already supported in https://infra.apache.org/release-signing.html#automated-release-signing by requesting a code-signing server from infra. I have not explored this myself, but this looks like it's the current approved approach. The sigstore approach looks good - my only concern is the asf approval part, which I have no insights about.

I really like the setup that you have for GH discussions to sync with the dev email list. As a last step, can the announcement discussion be auto-created triggered by pushing the release tag? the content should be templated and using highlights from the changelog.