Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support write #10

Merged
merged 18 commits into from
Dec 25, 2024
Merged

feat: support write #10

merged 18 commits into from
Dec 25, 2024

Conversation

ZENOTME
Copy link
Collaborator

@ZENOTME ZENOTME commented Dec 23, 2024

This PR:

  1. support converting arrow struct to iceberg struct
    feat: support arrow_struct_to_iceberg_struct apache/iceberg-rust#731
  2. support kinds of writer: fanout partition writer, sort_position_delete_writer, equality delta writer, precompute partition writer
  1. fix FieldSummary generated from Manifest (upstream merged)
  2. support serialize DataFile
  1. add delete file support for transaction
  1. fix nullable field in eq delete writer
  1. support to delete empty file in writer
  1. fix var len decimal of parquet stats

@ZENOTME ZENOTME requested review from chenzl25 and xxchan December 23, 2024 02:52
Copy link
Member

@xxchan xxchan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the link to upstream PR and what's the status? 👀

@ZENOTME
Copy link
Collaborator Author

ZENOTME commented Dec 23, 2024

What's the link to upstream PR and what's the status? 👀

I have add more context about this.

Copy link
Collaborator

@chenzl25 chenzl25 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Since risingwavelabs/risingwave#19887 passes all iceberg tests. I think it is ok to move fast to migrate from icelake to iceberg-rust.

@chenzl25 chenzl25 merged commit 2a97e6f into rw_rebase_main_20241129 Dec 25, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants