feat: support reading and writingStringView
and BinaryView
in parquet (part 1)
#5618
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
First part of #5530
Rationale for this change
This is the first 3 commits from #5557 by @ariesdevil for initial support of reading
StringViewArray
andBinaryViewArray
from parquetThe performance is non ideal (it copies string data several times) but it does include benchmarks and tests.
Thus I would like to merge this in and then continue iterating on the design in #5557 to keep the PRs smaller and more manageable
What changes are included in this PR?
StringViewArray
andBinaryViewArray
from parquetAre there any user-facing changes?
The arrow reader/writer can now read /write
StringViewArray
andBinaryViewArray
s