Skip to content

Commit

Permalink
refactor(parquet): Move arrow RleEncodingInternal to common (#11795)
Browse files Browse the repository at this point in the history
Summary:
Follow-up PR: #11711
Both Parquet reader and writer depend on RleEncodingInternal.
This currently resides in parquet/writer/arrow which introduces a dependency of the writer to the reader.
No functional changes.

CC: majetideepak pedroerp

Pull Request resolved: #11795

Reviewed By: DanielHunte

Differential Revision: D67152291

Pulled By: kgpai

fbshipit-source-id: a4a1cc49718e8258cb9e170c44320e04ed3ac581
  • Loading branch information
wypb authored and facebook-github-bot committed Dec 18, 2024
1 parent 2f81755 commit e3f7c5f
Show file tree
Hide file tree
Showing 13 changed files with 998 additions and 1,050 deletions.
4 changes: 2 additions & 2 deletions velox/dwio/common/tests/BitPackDecoderBenchmark.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
#include "velox/common/base/BitUtil.h"
#include "velox/common/base/Exceptions.h"
#include "velox/dwio/common/BitPackDecoder.h"
#include "velox/dwio/parquet/writer/arrow/util/BitStreamUtilsInternal.h"
#include "velox/dwio/parquet/common/BitStreamUtilsInternal.h"

#ifdef __BMI2__
#include "velox/dwio/common/tests/Lemire/bmipacking32.h"
Expand Down Expand Up @@ -307,7 +307,7 @@ void lemirebmi2(uint8_t bitWidth, uint32_t* result) {

template <typename T>
void arrowBitUnpack(uint8_t bitWidth, T* result) {
facebook::velox::parquet::arrow::bit_util::BitReader bitReader(
facebook::velox::parquet::BitReader bitReader(
reinterpret_cast<const uint8_t*>(bitPackedData[bitWidth].data()),
BYTES(kNumValues, bitWidth));
bitReader.GetBatch<T>(bitWidth, result, kNumValues);
Expand Down
Loading

0 comments on commit e3f7c5f

Please sign in to comment.