Skip to content

Commit

Permalink
[fix](hive) fix block decompressor bug (apache#45289)
Browse files Browse the repository at this point in the history
### What problem does this PR solve?

Problem Summary:
In the block decompressor, when it is found that the input data is less
than 4 bytes (the header size of the large block), should set
more_input_bytes instead of reporting an error.
  • Loading branch information
suxiaogang223 authored Dec 13, 2024
1 parent 735db65 commit c7fefc3
Showing 1 changed file with 12 additions and 12 deletions.
24 changes: 12 additions & 12 deletions be/src/exec/decompressor.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -492,15 +492,15 @@ Status Lz4BlockDecompressor::decompress(uint8_t* input, size_t input_len, size_t
auto* output_ptr = output;

while (input_len > 0) {
//if faild , fall back to large block begin
auto* large_block_input_ptr = input_ptr;
auto* large_block_output_ptr = output_ptr;

if (input_len < sizeof(uint32_t)) {
return Status::InvalidArgument(strings::Substitute(
"fail to do hadoop-lz4 decompress, input_len=$0", input_len));
*more_input_bytes = sizeof(uint32_t) - input_len;
break;
}

//if faild, fall back to large block begin
auto* large_block_input_ptr = input_ptr;
auto* large_block_output_ptr = output_ptr;

uint32_t remaining_decompressed_large_block_len = BigEndian::Load32(input_ptr);

input_ptr += sizeof(uint32_t);
Expand Down Expand Up @@ -609,15 +609,15 @@ Status SnappyBlockDecompressor::decompress(uint8_t* input, size_t input_len,
auto* output_ptr = output;

while (input_len > 0) {
//if faild , fall back to large block begin
auto* large_block_input_ptr = input_ptr;
auto* large_block_output_ptr = output_ptr;

if (input_len < sizeof(uint32_t)) {
return Status::InvalidArgument(strings::Substitute(
"fail to do hadoop-snappy decompress, input_len=$0", input_len));
*more_input_bytes = sizeof(uint32_t) - input_len;
break;
}

//if faild, fall back to large block begin
auto* large_block_input_ptr = input_ptr;
auto* large_block_output_ptr = output_ptr;

uint32_t remaining_decompressed_large_block_len = BigEndian::Load32(input_ptr);

input_ptr += sizeof(uint32_t);
Expand Down

0 comments on commit c7fefc3

Please sign in to comment.