Skip to content

Commit

Permalink
Fix DoraLoadCommandIntegrationTest
Browse files Browse the repository at this point in the history
### What changes are proposed in this pull request?

Fix DoraLoadCommandIntegrationTest.

### Why are the changes needed?

Without this PR, DoraLoadCommandIntegrationTest may fail with the following exception

```
[ERROR] alluxio.client.cli.fs.command.DoraLoadCommandIntegrationTest.testCommand  Time elapsed: 7.995 s  <<< FAILURE!
java.lang.AssertionError
        at org.junit.Assert.fail(Assert.java:87)
        at org.junit.Assert.assertTrue(Assert.java:42)
        at org.junit.Assert.assertTrue(Assert.java:53)
        at alluxio.client.cli.fs.command.DoraLoadCommandIntegrationTest.testCommand(DoraLoadCommandIntegrationTest.java:107)
```

The cause is that
- The second job is using the same path as the first job
- While submitting the second job, the second job is still in "cleaning" state. As the following log shows

    ```
    2024-01-31 09:52:33,057 [master-rpc-executor-TPE-thread-494] WARN  scheduler.Scheduler (Scheduler.java:submitJob) - There's concurrent submit while job is still in cleaning state
    ```

- "progress" returns the progress of the first job instead of the second one

The PR changes to use a different path for the second job, which avoids this issue.

### Does this PR introduce any user facing changes?

NO

			pr-link: #18504
			change-id: cid-331ba5508e86e8161006073d452ab1ba6230473a
  • Loading branch information
secfree authored Feb 1, 2024
1 parent e9a32c9 commit 07ec4b9
Showing 1 changed file with 16 additions and 14 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -84,24 +84,26 @@ public void testCommand() throws Exception {
assertTrue(mOutput.toString().contains("\"mVerbose\":true"));

// Test load with regx pattern file filter
createByteFileInUfs("/testRoot/testFileD", Constants.MB);
createByteFileInUfs("/testRoot/testDirectory/testFileE", Constants.MB);
createByteFileInUfs("/testRoot/testDirectory/testFileF", Constants.MB);
createByteFileInUfs("/testRoot/testDirectory/testFileG1", Constants.MB);
createByteFileInUfs("/testRoot/testDirectory/testFileG2", Constants.MB);

AlluxioURI uriD = new AlluxioURI("/testRoot/testFileD");
AlluxioURI uriE = new AlluxioURI("/testRoot/testDirectory/testFileE");
AlluxioURI uriF = new AlluxioURI("/testRoot/testDirectory/testFileF");
AlluxioURI uriG1 = new AlluxioURI("/testRoot/testDirectory/testFileG1");
AlluxioURI uriG2 = new AlluxioURI("/testRoot/testDirectory/testFileG2");
mTestFolder.newFolder("testRoot/testDirectory2");
createByteFileInUfs("/testRoot/testDirectory2/testFileD", Constants.MB);
createByteFileInUfs("/testRoot/testDirectory2/testFileE", Constants.MB);
createByteFileInUfs("/testRoot/testDirectory2/testFileF", Constants.MB);
createByteFileInUfs("/testRoot/testDirectory2/testFileG1", Constants.MB);
createByteFileInUfs("/testRoot/testDirectory2/testFileG2", Constants.MB);

AlluxioURI uriD = new AlluxioURI("/testRoot/testDirectory2/testFileD");
AlluxioURI uriE = new AlluxioURI("/testRoot/testDirectory2/testFileE");
AlluxioURI uriF = new AlluxioURI("/testRoot/testDirectory2/testFileF");
AlluxioURI uriG1 = new AlluxioURI("/testRoot/testDirectory2/testFileG1");
AlluxioURI uriG2 = new AlluxioURI("/testRoot/testDirectory2/testFileG2");

mOutput.reset();
assertEquals(0, mFsShell.run("load", path, "--submit",
String path2 = path + "/testDirectory2";
assertEquals(0, mFsShell.run("load", path2, "--submit",
"--file-filter-regx", ".*G[1|2]"));
assertEquals(0, mFsShell.run("load", path, "--progress"));
assertEquals(0, mFsShell.run("load", path2, "--progress"));
while (!mOutput.toString().contains("SUCCEEDED")) {
assertEquals(0, mFsShell.run("load", path, "--progress"));
assertEquals(0, mFsShell.run("load", path2, "--progress"));
Thread.sleep(1000);
}
assertTrue(mOutput.toString().contains("Inodes Processed: 2"));
Expand Down

0 comments on commit 07ec4b9

Please sign in to comment.