Skip to content

Commit

Permalink
Fix negative size if specified filesize over Integer.MAX_VALUE for st…
Browse files Browse the repository at this point in the history
…ressWorkerBench

### What changes are proposed in this pull request?

Change to use long type store file size.

### Why are the changes needed?

Without this PR, we cannot specified a filesize lagger than Integer.MAX_VALUE.

### Does this PR introduce any user facing changes?

No

			pr-link: #18492
			change-id: cid-2b816d1f2cbc9ebcf888b06802eb682fb76d55c2
  • Loading branch information
maobaolong authored Jan 19, 2024
1 parent 6f2a527 commit 97f325a
Showing 1 changed file with 2 additions and 2 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -155,7 +155,7 @@ public void prepare() throws Exception {

// initialize the base, for only the non-distributed task (the cluster launching task)
Path basePath = new Path(mParameters.mBasePath);
int fileSize = (int) FormatUtils.parseSpaceSize(mParameters.mFileSize);
long fileSize = FormatUtils.parseSpaceSize(mParameters.mFileSize);
int numFiles = getTotalFileNumber();

// Generate the file paths using the same heuristics so all nodes have the same set of paths
Expand Down Expand Up @@ -266,7 +266,7 @@ public void generateTestFilePaths(Path basePath) throws IOException {
LOG.info("{} file paths generated", mFilePaths.length);
}

private void prepareTestFiles(Path basePath, int fileSize, FileSystem prepareFs)
private void prepareTestFiles(Path basePath, long fileSize, FileSystem prepareFs)
throws IOException {
int numFiles = mFilePaths.length;
LOG.info("Preparing {} test files under {}", numFiles, basePath);
Expand Down

0 comments on commit 97f325a

Please sign in to comment.