Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: changes to rollout controller for eviction #959

Merged
merged 5 commits into from
Dec 30, 2024

Conversation

Arvindthiru
Copy link
Contributor

@Arvindthiru Arvindthiru commented Nov 15, 2024

Description of your changes

Fixes #

I have:

  • Run make reviewable to ensure this PR is ready for review.

How has this code been tested

Special notes for your reviewer

Copy link
Contributor

@ryanzhang-oss ryanzhang-oss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to add a few e2e test for the eviction cases

@Arvindthiru Arvindthiru marked this pull request as ready for review November 28, 2024 00:49
@Arvindthiru
Copy link
Contributor Author

Will add another PR to add more E2Es

@@ -607,6 +607,202 @@ var _ = Describe("Test the rollout Controller", func() {
}, 5*time.Minute, interval).Should(BeTrue(), "rollout controller should roll all the bindings to use the latest resource snapshot")
})

It("Rollout should be blocked, then unblocked by eviction - evict unscheduled binding", func() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what about the base case that a bound binding is evicted and scheduler creates another binding, make sure we rollout that binding?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added as an E2E test since it involves the entire flow

pkg/controllers/rollout/controller_test.go Outdated Show resolved Hide resolved
pkg/controllers/rollout/controller_test.go Outdated Show resolved Hide resolved
test/e2e/placement_eviction_test.go Outdated Show resolved Hide resolved
test/e2e/placement_eviction_test.go Outdated Show resolved Hide resolved
test/e2e/placement_eviction_test.go Show resolved Hide resolved
test/e2e/placement_eviction_test.go Outdated Show resolved Hide resolved
@Arvindthiru Arvindthiru marked this pull request as draft December 17, 2024 19:09
@Arvindthiru Arvindthiru marked this pull request as ready for review December 18, 2024 23:17
pkg/controllers/rollout/controller_integration_test.go Outdated Show resolved Hide resolved
@@ -205,23 +206,23 @@ func TestWaitForResourcesToCleanUp(t *testing.T) {
"test deleting binding block schedule binding on the same cluster": {
allBindings: []*fleetv1beta1.ClusterResourceBinding{
generateClusterResourceBinding(fleetv1beta1.BindingStateScheduled, "snapshot-1", cluster1),
generateDeletingClusterResourceBinding(cluster1),
setDeletionTimeStampForBinding(generateClusterResourceBinding(fleetv1beta1.BindingStateUnscheduled, "snapshot-1", cluster1)),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why add the delete timestamp here? The function it tests doesn't care.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pkg/controllers/rollout/controller_test.go Show resolved Hide resolved
pkg/controllers/rollout/controller_test.go Show resolved Hide resolved
pkg/controllers/rollout/controller_test.go Outdated Show resolved Hide resolved
},
},
wantTobeUpdatedBindings: []int{2}, // specified MaxSurge helps us pick one scheduled binding to rollout. we don't have any ready binding so we don't remove any binding.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In theory, we pick one from 2 or 3. is it guaranteed to pick 2?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's guaranteed we pick 2 before 3, we create bounding candidates based on order from allBindings https://github.com/Azure/fleet/blob/main/pkg/controllers/rollout/controller.go#L380 and then we chose update bindings from bounding candidates in the same order https://github.com/Azure/fleet/blob/main/pkg/controllers/rollout/controller.go#L477

pkg/controllers/rollout/controller_test.go Outdated Show resolved Hide resolved
pkg/controllers/rollout/controller_test.go Outdated Show resolved Hide resolved
pkg/controllers/rollout/controller.go Show resolved Hide resolved
pkg/controllers/rollout/controller_test.go Outdated Show resolved Hide resolved
Comment on lines +31 to +43
// Create the CRP.
crp := &placementv1beta1.ClusterResourcePlacement{
ObjectMeta: metav1.ObjectMeta{
Name: crpName,
// Add a custom finalizer; this would allow us to better observe
// the behavior of the controllers.
Finalizers: []string{customDeletionBlockerFinalizer},
},
Spec: placementv1beta1.ClusterResourcePlacementSpec{
ResourceSelectors: workResourceSelector(),
},
}
Expect(hubClient.Create(ctx, crp)).To(Succeed(), "Failed to create CRP %s", crpName)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there is a createCRP function in test/e2e/utils_test.go

createWorkResources()

// Create the CRP.
crp := &placementv1beta1.ClusterResourcePlacement{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

@Arvindthiru Arvindthiru merged commit b000bd9 into Azure:main Dec 30, 2024
12 checks passed
@Arvindthiru Arvindthiru mentioned this pull request Dec 31, 2024
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants