-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Hash binary values #3098
Hash binary values #3098
Conversation
Codecov Report
@@ Coverage Diff @@
## master #3098 +/- ##
==========================================
- Coverage 85.95% 85.92% -0.03%
==========================================
Files 290 290
Lines 52260 52285 +25
==========================================
+ Hits 44919 44928 +9
- Misses 7341 7357 +16
📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
DataType::Binary => { | ||
hash_array!( | ||
BinaryArray, | ||
col, | ||
&[u8], | ||
hashes_buffer, | ||
random_state, | ||
multi_col | ||
); | ||
} | ||
DataType::LargeBinary => { | ||
hash_array!( | ||
LargeBinaryArray, | ||
col, | ||
&[u8], | ||
hashes_buffer, | ||
random_state, | ||
multi_col | ||
); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Thanks @Dandandan
FYI @waitingkuo |
Benchmark runs are scheduled for baseline = b80c853 and contender = b6832d5. b6832d5 is a master commit associated with this PR. Results will be available as each benchmark for each run completes. |
Which issue does this PR close?
Closes #3050
Rationale for this change
Currently we can not group by binary values
What changes are included in this PR?
Implementing hashing for binary values. Also some first casting support (but depends on apache/arrow-rs#2402 to be impemented).
Are there any user-facing changes?