Tracking issues of iceberg rust v0.4.0 Release

[sungwy]

This issue is used to track tasks of the iceberg rust 0.4.0 release.

Tasks

Blockers

Blockers are the tasks that must be completed before the release.

• Bug in PartialEq for Struct #706
• Why shouldn't we return an UnboundPartitionSpec instead? #694
• feat: Allow reading data type Decimal #663

The remaining issues listed under 0.4.0 Milestone do not seem like blockers, and could be tracked in the next release.

Build Release

GitHub Side

• Bump version in project: Prep 0.4.0 release #809
• Update docs
• Generate dependencies list
• Push release candidate tag to GitHub

ASF Side

• Create an ASF Release
• Upload artifacts to the SVN dist repo

Voting

• Start VOTE at iceberg community

Official Release

• Push the release git tag
• Verify iceberg-rust release in Cargo and pyiceberg-core releases in PyPi
• Publish artifacts to SVN RELEASE branch
• Change Iceberg Rust Website download link
• Send the announcement

For details of each step, please refer to: https://rust.iceberg.apache.org/release

[Xuanwo]

I believe we need something like https://github.com/apache/opendal/blob/main/scripts/release.py to help generate ASF source tarballs for core and python binding.

[c-thiel]

@sungwy I think we should add #694 to the blockers as well.
If we decide to not introduce a SchemalessPartitionSpec, we shouldn't release it as part of 0.4.0 as it would be a significant change to revert it.

[sungwy]

@c-thiel Hi Christian, thanks for flagging the issue! I've been tracking the discussion, and it looks like we are getting closer to a consensus on how we'd like to resolve the issue.

I will ensure the resolution gets taken into account within this release.

[Xuanwo]

It's better to include #665 in. I'm working on this now.

[Xuanwo]

@sungwy I think we should add #694 to the blockers as well. If we decide to not introduce a SchemalessPartitionSpec, we shouldn't release it as part of 0.4.0 as it would be a significant change to revert it.

Hi @c-thiel, are you working on this now? I hope we can start the release process next week before Christmas; otherwise, we might have to see the new release until next year.

[c-thiel]

@sungwy I think we should add #694 to the blockers as well. If we decide to not introduce a SchemalessPartitionSpec, we shouldn't release it as part of 0.4.0 as it would be a significant change to revert it.

Hi @c-thiel, are you working on this now? I hope we can start the release process next week before Christmas; otherwise, we might have to see the new release until next year.

@Xuanwo
I am talking with @Fokko about this today. We'll try to come to a conclusion and post our findings in the issue.

[c-thiel]

@Xuanwo, @sungwy #694 is either fixed by leaving things as-is or by merging #771 - see my comment at the top of the PR.

[Xuanwo]

@Xuanwo, @sungwy #694 is either fixed by leaving things as-is or by merging #771 - see my comment at the top of the PR.

Thank you @c-thiel and @Fokko very much for working on this! 🙏

[sungwy]

Thank you for working on the issue @c-thiel and keeping this thread updated!

[Xuanwo]

It's better to include #665 in. I'm working on this now.

Update: #665 is very close to get merging in.

[sungwy]

Also update on #706 : I've investigated it and it looks like a non-issue. I'm waiting on the issue reporters to respond before closing it out

[sungwy]

It looks like two issues have been resolved, and #771 is very close to being merged.

We'll get started with the release shortly! 🚀

I believe we need something like https://github.com/apache/opendal/blob/main/scripts/release.py to help generate ASF source tarballs for core and python binding.

Hi @Xuanwo - I started taking a look into this, and it looks like the existing release.sh script would already create a tarball including all the source code for the core rust crates, as well as those of the python bindings.

The opendal reference implementation creates different tarballs for each binding and integrations separately, all including the the source code of the rust core crates as well.

Since we have decided to have a single coupled release schedule for the rust core crates as well as the associated bindings, I'm wondering if it would just make sense to keep the source codes in the same tarball to simplify the process. We could also increment the version of pyiceberg_core library to 0.4.0 if that would make more sense for the tightly coupled release schedule.

I looked up the ASF release policy, and I couldn't find any recommendations on how we should organize the source code for sub-distributions. I do see guidelines for how the source code needs to be uploaded and signed, and that the name of the packages will need to be prefixed by apache and the project name. I believe this will be the case whether we group the source code in a single tarball, or into separate ones.

I'm curious to hear your thoughts on this suggestion @Xuanwo @Fokko and @liurenjie1024 do you think uploading the source package as a single artifact would make sense in our case?

[Fokko]

I've cleaned up the milestone, and I think we're good to go 🚀

@sungwy Since we release everything simultaneously, I would highly recommend putting everything in a single tarbal to keep things simple.

[Xuanwo]

Hi, @sungwy, I prefer to split them into different tarballs since they have different versions. However, I don't have a strong opinion on this. I believe it would be great to release them first and then refine the process in the next release.

[liurenjie1024]

Should we cut a branch for 0.4.0-rc1?

[ZENOTME]

Should we merge #795 before release?

[Xuanwo]

Hi, I personally believe #795 is not a blocker. Let's proceed with the release and avoid adding more PRs unless we have truly critical issues to address.

What do you think? cc @ZENOTME

---

Sorry if I put too much burden here, but I really want to finalize a release before 2025 arrives. As I mentioned in previous comments, this week is the only one before Christmas when we can make it happen.

[ZENOTME]

Both look good to me. According to #795 (comment), this reorder will not happen if we use in avro format. In this release version, we don't expose the interface to let the user get the serializable representation, so I think this issue will not affect the user in this version.🤔

[Xuanwo]

Both look good to me. According to #795 (comment), this reorder will not happen if we use in avro format. In this release version, we don't expose the interface to let the user get the serializable representation, so I think this issue will not affect the user in this version.🤔

Thank you for your understanding. Let's continue addressing this issue without delaying the 0.4.0 release.

[sungwy]

Sounds good folks. I'll get started with the release.

Based on the discussion above, I think we are good to start the release, with the new Python binding included in the same source package.

Could someone help review this PR so we can get the versions synced to 0.4.0? #808

[Xuanwo]

Could someone help review this PR so we can get the versions synced to 0.4.0? #808

Merged. Let's move!

[sungwy]

Let's get this party started~! 🎈

Here's the PR to remove deprecated functions, and bump the version of iceberg-rust for review: #809

[Xuanwo]

Hi, @sungwy, #826 has been fixed. Would you like to start a new rc instead?

[sungwy]

Yes, I'm on it 🖖