Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[CELEBORN-1625] Add parameter
skipCompress
for pushOrMergeData
### What changes were proposed in this pull request? Add new parameters to the pushOrMergeData method for gluten ### Why are the changes needed? Currently, in the scenario of Gluten Fallback, it is possible for a stage to have both Native and Java shuffles simultaneously. Since the ShuffleClient is a singleton, the compression settings for the ShuffleClient will only utilize either the Native or Java configuration. By adding new parameters, we aim to allow Gluten to control whether compression should be applied in this scenario. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? CI Closes #2769 from kerwin-zk/CELEBORN-1625. Authored-by: xiyu.zk <[email protected]> Signed-off-by: zky.zhoukeyong <[email protected]>
- Loading branch information