Skip to content

Commit

Permalink
fix(backup): Fix the stack overflow when read large sst file (apache#…
Browse files Browse the repository at this point in the history
…2059)

After refactoring to use RocksDB APIs to read files from local
filesystem, it may cause stack overflow when the file to read
is larger than the stack size (say 8MB).

This patch changes to use heap instead of stack to store the
file content.
  • Loading branch information
acelyc111 authored Jul 4, 2024
1 parent efe950d commit 7f50394
Showing 1 changed file with 3 additions and 2 deletions.
5 changes: 3 additions & 2 deletions src/block_service/hdfs/hdfs_service.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@
#include "utils/fmt_logging.h"
#include "utils/safe_strerror_posix.h"
#include "utils/strings.h"
#include "utils/utils.h"

DSN_DEFINE_uint64(replication,
hdfs_read_batch_size_bytes,
Expand Down Expand Up @@ -435,8 +436,8 @@ dsn::task_ptr hdfs_file_object::upload(const upload_request &req,
}

rocksdb::Slice result;
char scratch[file_size];
s = rfile->Read(file_size, &result, scratch);
auto scratch = dsn::utils::make_shared_array<char>(file_size);
s = rfile->Read(file_size, &result, scratch.get());
if (!s.ok()) {
LOG_ERROR(
"read local file '{}' failed, err = {}", req.input_local_name, s.ToString());
Expand Down

0 comments on commit 7f50394

Please sign in to comment.