48.0.1 arrow patch release

[alamb]

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
As noted by @wjones127 on the DataFusion 33 release thread: https://lists.apache.org/thread/dwdk1yogq65d4vnfcjkjh02c9mbqmojp

There was a regression #5038 introduced in the 48.0.0 release related to parsing / serializing large integer values in JSON

The fix #5042 will be included in 49.0.0. However, if we release software (like DataFusion 33.0.0) that depends on 48.0.0 users will have no way to avoid the bug if they also upgrade

Describe the solution you'd like
I would like to consider doing a separate release 48.1.0 that includes the fix for the bugs above

Describe alternatives you've considered
We could alternately delay releasing DataFusion 33 until we have released arrow 49.0.0 and updated DataFusion to use that

Additional context

Release Checklist

• Backport CI fixes to 48.0.0_maintenance #5061
• Backport: Fix RowSelection::intersection (#5036) (#5041) to 48.0.0 maintenance #5060
• Backport: Fix serialization of large integers (#5038) (#5042) to 48.0.0_maintenance #5059
• Make CHANGELOG + release notes: Update version to 48.0.1 and add README #5062
• Start vote thread : https://lists.apache.org/thread/4vbh1vd39vlty8ck3j3ddzrqxhd9ybm9
• Approved and Publish to crates.io: https://lists.apache.org/thread/s3h1q88xlc4mwx7fn5b6pnwytd15t6o8

[alamb]

I believe @tustvold would prefer the "hold the datafusion release" approach

@andygrove do you have a preference?

I would be happy to help with either approach

[andygrove]

I am fine with delaying the DataFusion release. I would like to merge apache/datafusion#8072 first anyway (once I have verified that it fixes apache/datafusion#8069)

[wjones127]

I feel silly for having forgotten the possibility of a patch release. Also, by patch release, don't you mean 48.0.1?

If DataFusion waits, then downstream projects I know of won't use the patch release. So the only reason then is if there are other users who might care. I'm ambivalent at the moment, but I think in the future we could consider more immediately patch releases when we find sufficiently serious bugs.

[alamb]

I feel silly for having forgotten the possibility of a patch release. Also, by patch release, don't you mean 48.0.1?

If it is just a bug fix that is a good point.

[alamb]

but I think in the future we could consider more immediately patch releases when we find sufficiently serious bugs.

I think the limiting factor here has always been maintainer bandwidth to make the releases

[alamb]

I believe that having a 48.0.1 patch release will be good for several reasons:

1. Existing users of 48.0.0 can get the fix
2. It will disconnect the DataFusion 33.0.0 release from API changes needed to support the arrow 49.0.0 (and object_store 0.8.0 upgrade) which is still under discussion (e.g. Implement StreamTable and StreamTableProvider (#7994) datafusion#8021)

Therefore I plan to make a release candidate later today

[alamb]

Since I am going to do 48.0.0 patch release anyways, I am going to include the fix for #5036, which @b4l hit in DataFusion as well: apache/datafusion#8092 (we also hit it in InfluxData earlier in the week)

[alamb]

Update here is I have prepared a branch and backported the relevant fixes (thanks @viirya for the quick reviews 🙏 ): https://github.com/apache/arrow-rs/tree/48.0.0_maintenance

I have also sent a note to the dev list: https://lists.apache.org/thread/r7bgll8zr2fgvob6mqyqfv7c99bbfb3p

I plan to make a release candidate and start a vote thread later today.

[andygrove]

Thanks @alamb. Decoupling this from upgrading DF to use arrow 49 makes sense. There are quite a few changes.

I am generally in favor of the patch release approach, but as you said, maintainer bandwidth is an issue.

[viirya]

Thanks @alamb for working on the patch release.

[alamb]

I have started a vote thread: https://lists.apache.org/thread/4vbh1vd39vlty8ck3j3ddzrqxhd9ybm9

[alamb]

The release was approved and is available now on crates.io https://lists.apache.org/thread/s3h1q88xlc4mwx7fn5b6pnwytd15t6o8

[tustvold]

label_issue.py automatically added labels {'arrow'} from #5059

[tustvold]

label_issue.py automatically added labels {'cherry-picked'} from #5059

[tustvold]

label_issue.py automatically added labels {'parquet'} from #5060