Release arrow-rs / parquet minor version 54.1.0 (Jan 2025)

[alamb]

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

Our release schedule is here https://github.com/apache/arrow-rs?tab=readme-ov-file#release-versioning-and-schedule

• We released 54.0.0 in Dec 2024 Release arrow-rs / parquet major version 54.0.0 (December 2024) #6342

Items to fix before the release:

• Arrow IPC Writer Panics for sliced nested arrays #6997
• Verify with downstream DataFusion tests: WIP: Update to arrow-rs / parquet 54.1.0 datafusion#14328

Describe the solution you'd like

Create a 54.1.0 release:

• Release notes and bump version: Prepare for 54.1.0 release #7031
• Release Candidate + Vote thread: https://lists.apache.org/thread/s57m8ohovyf36gtp53623jmhwr0q3y95
• Release to crates.io / approval: https://lists.apache.org/thread/xqpno65v0l3m1gtffg7btgmgynkqbw2w

Describe alternatives you've considered

Additional context

[alamb]

I plan to make this release next week

[alamb]

I added #6997 as an item we should probably fix before release

[alamb]

I plan to make the release candidate early next week (of Jan 27)

[alamb]

@kou suggests on apache/arrow-site#580 (comment):

+1

@alamb It may be better that arrow-rs creates GibHub Releases (https://github.com/apache/arrow-rs/releases) with the generated changelog. FYI: We can create a GitHub Releases by gh release create ${GITHUB_REF_NAME} --notes-file CHANGELOG.md (with env: GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}) by GItHub Actions automatically. See also https://github.com/apache/arrow-java/blob/main/.github/workflows/release.yml .

I will review / see if we can do that for the 54.1.0 release

[alamb]

I have created a PR for version + CHANGELOG:

• Prepare for 54.1.0 release #7031

I would love to get this PR from @Chen-Yuan-Lai in too:

• Add is_valid and truncate methods to NullBufferBuilder #7013

But strictly speaking it is not blocking the release

[alamb]

@kou suggests on apache/arrow-site#580 (comment):

FYI @kou -- I made github releases for 53.0.0 to 54.0.0 in github. It was straightforward:

I have added instructions on how to do so for future relases in

• Prepare for 54.1.0 release #7031

[alamb]

I have found what looks like a regression on main. See details on

• WIP: Update to arrow-rs / parquet 54.1.0 datafusion#14328 (comment)
• Seems like it is realted to feat(arrow-select): concat kernel will merge dictionary values for list of dictionaries #6893

[kou]

Thanks!

FYI: We can automate the GitHub Releases creation process by GitHub Actions. If it helps arrow-rs release process, I can open a PR for it. :-)

[alamb]

Thanks!

FYI: We can automate the GitHub Releases creation process by GitHub Actions. If it helps arrow-rs release process, I can open a PR for it. :-)

Yes that would be amazing. Thank you @kou

[alamb]

Update on release status is I filed a ticket for the regression and @tustvold and @rluvaton have kindly already reviewed it

• Regression: Concatenating sliced ListArrays is broken #7034

[kou]

FYI: We can automate the GitHub Releases creation process by GitHub Actions. If it helps arrow-rs release process, I can open a PR for it. :-)

Yes that would be amazing. Thank you @kou

OK!

• Create GitHub releases automatically on tagging #7041
• Create GitHub releases automatically on tagging #7042

[alamb]

I have created a release candidate and started voting:

• https://lists.apache.org/thread/s57m8ohovyf36gtp53623jmhwr0q3y95

[kou]

It's not related to GitHub releases but could you remove RCs from https://dist.apache.org/repos/dist/dev/arrow/ after each release vote passed?

I thought that it's explained in a ASF document but I couldn't find it... For example, https://infra.apache.org/release-publishing.html doesn't mention it...

FYI:

• apache/arrow uses https://github.com/apache/arrow/blob/main/dev/release/post-09-remove-old-artifacts.sh for it
• apache/arrow-adbc uses a similar script: https://github.com/apache/arrow-adbc/blob/main/dev/release/post-10-remove-old-artifacts.sh
• apache/arrow-go uses GitHub releases instead of https://dist.apache.org/repos/dist/dev/arrow/ for distributing RCs. So it doesn't need to remove old RCs: https://github.com/apache/arrow-go/blob/main/dev/release/release.sh

I can help this too. :-)

[alamb]

It's not related to GitHub releases but could you remove RCs from https://dist.apache.org/repos/dist/dev/arrow/ after each release vote passed?

Sure sounds like a great idea

I can help this too. :-)

Thanks @kou !

The script we use to remove old releases is here https://github.com/apache/arrow-rs/blob/main/dev/release/remove-old-releases.sh (it gets run at the end of the release process)

[kou]

Here you are:

• Remove all RCs after release #7059
• Remove all RCs after release #7060

[alamb]

https://lists.apache.org/thread/xqpno65v0l3m1gtffg7btgmgynkqbw2w

---------- Forwarded message ---------
From: Andrew Lamb [email protected]
Date: Sun, Feb 2, 2025 at 6:23 AM
Subject: [RESULT] [VOTE][RUST] Release Apache Arrow Rust 54.1.0 RC1
To: dev [email protected]

With 4 +1 votes (3 binding) the release is approved.

The release is available here:
https://dist.apache.org/repos/dist/release/arrow/arrow-rs-54.1.0

It is also available as a github release (thanks Kou for the automation):
https://github.com/apache/arrow-rs/releases/54.1.0

I have also uploaded it to crates.io

Thanks again everyone for the help,
Andrew