You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
@rdblue pointed me to https://github.com/airlift/aircompressor which provides non-native implementations of compression codecs. It claims to be much faster than native wrappers that parquet uses. This Jira is to track the work needed for exploring using these codecs, getting benchmark results and making changes including not needing to pool compressors and decompressors anymore. Note that this doesn't include SNAPPY since Parquet already has its own non-hadoopy implementation for it.
@rdblue pointed me to https://github.com/airlift/aircompressor which provides non-native implementations of compression codecs. It claims to be much faster than native wrappers that parquet uses. This Jira is to track the work needed for exploring using these codecs, getting benchmark results and making changes including not needing to pool compressors and decompressors anymore. Note that this doesn't include SNAPPY since Parquet already has its own non-hadoopy implementation for it.
Reporter: Samarth Jain / @samarthjain
Assignee: Samarth Jain / @samarthjain
PRs and other links:
Note: This issue was originally created as PARQUET-1643. Please see the migration documentation for further details.
The text was updated successfully, but these errors were encountered: