Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test/region_syncer: make TestPrepareCheckerWithTransferLeader more stable #8357

Merged
merged 2 commits into from
Jul 4, 2024
Merged

test/region_syncer: make TestPrepareCheckerWithTransferLeader more stable #8357

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
db60750
fix test
HuSharp Jul 3, 2024
7351cbe
Merge branch 'master' into fix_prepare_checker_tesy
ti-chi-bot[bot] Jul 4, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -254,7 +254,7 @@ basic-test: install-tools

ci-test-job: install-tools dashboard-ui pd-ut
@$(FAILPOINT_ENABLE)
./scripts/ci-subtask.sh $(JOB_COUNT) $(JOB_INDEX) || { $(FAILPOINT_DISABLE); exit 1; }
./scripts/ci-subtask.sh $(JOB_INDEX) || { $(FAILPOINT_DISABLE); exit 1; }
@$(FAILPOINT_DISABLE)

TSO_INTEGRATION_TEST_PKGS := $(PD_PKG)/tests/server/tso
Expand Down
2 changes: 1 addition & 1 deletion scripts/ci-subtask.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
#!/usr/bin/env bash

# ./ci-subtask.sh <TOTAL_TASK_N> <TASK_INDEX>
# ./ci-subtask.sh <TASK_INDEX>

ROOT_PATH_COV=$(pwd)/covprofile
ROOT_PATH_JUNITFILE=$(pwd)/junitfile
Expand Down
16 changes: 12 additions & 4 deletions tests/server/region_syncer/region_syncer_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -234,11 +234,13 @@ func TestPrepareChecker(t *testing.T) {
re.NoError(err)
err = pd2.Run()
re.NoError(err)
re.NotEmpty(cluster.WaitLeader())
// waiting for synchronization to complete
time.Sleep(3 * time.Second)
leaderServer = cluster.GetLeaderServer()
err = cluster.ResignLeader()
re.NoError(err)
re.Equal("pd2", cluster.WaitLeader())
re.NotEqual(leaderServer.GetServer().Name(), cluster.WaitLeader())
leaderServer = cluster.GetLeaderServer()
rc = leaderServer.GetServer().GetRaftCluster()
for _, region := range regions {
Expand Down Expand Up @@ -282,16 +284,22 @@ func TestPrepareCheckerWithTransferLeader(t *testing.T) {
re.NoError(err)
err = pd2.Run()
re.NoError(err)
re.NotEmpty(cluster.WaitLeader())
// waiting for synchronization to complete
time.Sleep(3 * time.Second)
leaderServer = cluster.GetLeaderServer()
err = cluster.ResignLeader()
re.NoError(err)
re.Equal("pd2", cluster.WaitLeader())
re.NotEqual(leaderServer.GetServer().Name(), cluster.WaitLeader())
rc = cluster.GetLeaderServer().GetRaftCluster()
re.True(rc.IsPrepared())

// transfer leader to pd1, can start coordinator immediately.
// transfer leader, can start coordinator immediately.
leaderServer = cluster.GetLeaderServer()
err = cluster.ResignLeader()
re.NoError(err)
re.Equal("pd1", cluster.WaitLeader())
re.NotEqual(leaderServer.GetLeader().GetName(), cluster.WaitLeader())
Copy link
Member

@rleungx rleungx Jul 3, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if the WaitLeader still returns ""?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we don't get a reasonable leader, maybe we can assume the test has failed?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The previous failure is caused by no leader and after this PR, will it still fail?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a detailed explanation in
#8351 (comment)

I think in this extreme situation, we can fail directly.

Copy link
Member Author

@HuSharp HuSharp Jul 4, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I created an issue to focus on the Waitleader failure situation :) #8359

rc = cluster.GetLeaderServer().GetServer().GetRaftCluster()
re.True(rc.IsPrepared())
re.NoError(failpoint.Disable("github.com/tikv/pd/pkg/schedule/changeCoordinatorTicker"))
}