Skip to content

Commit

Permalink
interface: add the new takeover option never, plus some minor changes (…
Browse files Browse the repository at this point in the history 
…#984)
  • Loading branch information
@michaelawyu
michaelawyu authored Dec 11, 2024
1 parent 0fb234b commit 9a3cd78
Show file tree
Hide file tree
Showing 11 changed files with 294 additions and 61 deletions.
82 changes: 69 additions & 13 deletions apis/placement/v1beta1/clusterresourceplacement_types.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -535,8 +535,19 @@ type ApplyStrategy struct {
// differences. No actual apply ops would be executed, and resources will be left alone as they
// are on the member clusters.
//
// If configuration differences are found on a resource, Fleet will consider this as an apply
// error, which might block rollout depending on the specified rollout strategy.
//
// Use ComparisonOption setting to control how the difference is calculated.
//
// ClientSideApply and ServerSideApply apply strategies only work when Fleet can assume
// ownership of a resource (e.g., the resource is created by Fleet, or Fleet has taken over
// the resource). See the comments on the WhenToTakeOver field for more information.
// ReportDiff apply strategy, however, will function regardless of Fleet's ownership
// status. One may set up a CRP with the ReportDiff strategy and the Never takeover option,
// and this will turn Fleet into a detection tool that reports only configuration differences
// but do not touch any resources on the member cluster side.
//
// For a comparison between the different strategies and usage examples, refer to the
// Fleet documentation.
//
Expand All @@ -545,10 +556,14 @@ type ApplyStrategy struct {
// +kubebuilder:validation:Optional
Type ApplyStrategyType `json:"type,omitempty"`

// AllowCoOwnership defines whether to apply the resource if it already exists in the target cluster and is not
// solely owned by fleet (i.e., metadata.ownerReferences contains only fleet custom resources).
// If true, apply the resource and add fleet as a co-owner.
// If false, leave the resource unchanged and fail the apply.
// AllowCoOwnership controls whether co-ownership between Fleet and other agents are allowed
// on a Fleet-managed resource. If set to false, Fleet will refuse to apply manifests to
// a resource that has been owned by one or more non-Fleet agents.
//
// Note that Fleet does not support the case where one resource is being placed multiple
// times by different CRPs on the same member cluster. An apply error will be returned if
// Fleet finds that a resource has been owned by another placement attempt by Fleet, even
// with the AllowCoOwnership setting set to true.
AllowCoOwnership bool `json:"allowCoOwnership,omitempty"`

// ServerSideApplyConfig defines the configuration for server side apply. It is honored only when type is ServerSideApply.
Expand All @@ -571,8 +586,9 @@ type ApplyStrategy struct {
//
// * IfNoDiff: with this action, Fleet will apply the hub cluster manifests to the member
// clusters if (and only if) pre-existing resources look the same as the hub cluster manifests.
//
// This is a safer option as pre-existing resources that are inconsistent with the hub cluster
// manifests will not be overwritten; in fact, Fleet will ignore them until the inconsistencies
// manifests will not be overwritten; Fleet will ignore them until the inconsistencies
// are resolved properly: any change you make to the hub cluster manifests would not be
// applied, and if you delete the manifests or even the ClusterResourcePlacement itself
// from the hub cluster, these pre-existing resources would not be taken away.
Expand All @@ -597,8 +613,24 @@ type ApplyStrategy struct {
// If appropriate, you may also delete the object from the member cluster; Fleet will recreate
// it using the hub cluster manifest.
//
// * Never: with this action, Fleet will not apply a hub cluster manifest to the member
// clusters if there is a corresponding pre-existing resource. However, if a manifest
// has never been applied yet; or it has a corresponding resource which Fleet has assumed
// ownership, apply op will still be executed.
//
// This is the safest option; one will have to remove the pre-existing resources (so that
// Fleet can re-create them) or switch to a different
// WhenToTakeOver option before Fleet starts processing the corresponding hub cluster
// manifests.
//
// If you prefer Fleet stop processing all manifests, use this option along with the
// ReportDiff apply strategy type. This setup would instruct Fleet to touch nothing
// on the member cluster side but still report configuration differences between the
// hub cluster and member clusters. Fleet will not give up ownership
// that it has already assumed though.
//
// +kubebuilder:default=Always
// +kubebuilder:validation:Enum=Always;IfNoDiff
// +kubebuilder:validation:Enum=Always;IfNoDiff;Never
// +kubebuilder:validation:Optional
WhenToTakeOver WhenToTakeOverType `json:"whenToTakeOver,omitempty"`
}
Expand Down Expand Up @@ -676,14 +708,35 @@ type ServerSideApplyConfig struct {
type WhenToTakeOverType string

const (
// WhenToTakeOverTypeIfNoDiff will apply manifests from the hub cluster only if there is no difference
// between the current resource snapshot version on the hub cluster and the existing
// resources on the member cluster. Otherwise, we will report the difference.
// WhenToTakeOverTypeIfNoDiff instructs Fleet to apply a manifest with a corresponding
// pre-existing resource on a member cluster if and only if the pre-existing resource
// looks the same as the manifest. Should there be any inconsistency, Fleet will skip
// the apply op; no change will be made on the resource and Fleet will not claim
// ownership on it.
//
// Note that this will not stop Fleet from processing other manifests in the same
// placement that do not concern the takeover process (e.g., the manifests that have
// not been created yet, or that are already under the management of Fleet).
WhenToTakeOverTypeIfNoDiff WhenToTakeOverType = "IfNoDiff"

// WhenToTakeOverTypeAlways will always apply the resource to the member cluster regardless
// if there are differences between the resource on the hub cluster and the existing resources on the member cluster.
// WhenToTakeOverTypeAlways instructs Fleet to always apply manifests to a member cluster,
// even if there are some corresponding pre-existing resources. Some fields on these
// resources might be overwritten, and Fleet will claim ownership on them.
WhenToTakeOverTypeAlways WhenToTakeOverType = "Always"

// WhenToTakeOverTypeNever instructs Fleet to never apply a manifest to a member cluster
// if there is a corresponding pre-existing resource.
//
// Note that this will not stop Fleet from processing other manifests in the same placement
// that do not concern the takeover process (e.g., the manifests that have not been created
// yet, or that are already under the management of Fleet).
//
// If you would like Fleet to stop processing manifests all together and do not assume
// ownership on any pre-existing resources, use this option along with the ReportDiff
// apply strategy type. This setup would instruct Fleet to touch nothing on the member
// cluster side but still report configuration differences between the hub cluster
// and member clusters. Fleet will not give up ownership that it has already assumed, though.
WhenToTakeOverTypeNever WhenToTakeOverType = "Never"
)

// +enum
Expand Down Expand Up @@ -978,8 +1031,11 @@ type DiffedResourcePlacement struct {

// TargetClusterObservedGeneration is the generation of the resource on the target cluster
// that contains the configuration differences.
// +kubebuilder:validation:Required
TargetClusterObservedGeneration int64 `json:"targetClusterObservedGeneration"`
//
// This might be nil if the resource has not been created yet on the target cluster.
//
// +kubebuilder:validation:Optional
TargetClusterObservedGeneration *int64 `json:"targetClusterObservedGeneration"`

// FirstDiffedObservedTime is the first time the resource on the target cluster is
// observed to have configuration differences.
Expand Down
13 changes: 7 additions & 6 deletions apis/placement/v1beta1/work_types.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -102,14 +102,14 @@ type WorkResourceIdentifier struct {
// Kind is the kind of the resource.
Kind string `json:"kind,omitempty"`

// Resource is the resource type of the resource
// Resource is the resource type of the resource.
Resource string `json:"resource,omitempty"`

// Namespace is the namespace of the resource, the resource is cluster scoped if the value
// is empty
// is empty.
Namespace string `json:"namespace,omitempty"`

// Name is the name of the resource
// Name is the name of the resource.
Name string `json:"name,omitempty"`
}

Expand All @@ -124,7 +124,6 @@ type DriftDetails struct {

// ObservedInMemberClusterGeneration is the generation of the applied manifest on the member
// cluster side.
//
// +kubebuilder:validation:Required
ObservedInMemberClusterGeneration int64 `json:"observedInMemberClusterGeneration"`

Expand Down Expand Up @@ -157,8 +156,10 @@ type DiffDetails struct {
// ObservedInMemberClusterGeneration is the generation of the applied manifest on the member
// cluster side.
//
// +kubebuilder:validation:Required
ObservedInMemberClusterGeneration int64 `json:"observedInMemberClusterGeneration"`
// This might be nil if the resource has not been created yet in the member cluster.
//
// +kubebuilder:validation:Optional
ObservedInMemberClusterGeneration *int64 `json:"observedInMemberClusterGeneration"`

// FirsftDiffedObservedTime is the timestamp when the configuration difference
// was first detected.
Expand Down
10 changes: 10 additions & 0 deletions apis/placement/v1beta1/zz_generated.deepcopy.go
View file

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 3 additions & 3 deletions config/crd/bases/placement.kubernetes-fleet.io_appliedworks.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -189,20 +189,20 @@ spec:
description: Kind is the kind of the resource.
type: string
name:
description: Name is the name of the resource
description: Name is the name of the resource.
type: string
namespace:
description: |-
Namespace is the namespace of the resource, the resource is cluster scoped if the value
is empty
is empty.
type: string
ordinal:
description: |-
Ordinal represents an index in manifests list, so the condition can still be linked
to a manifest even though manifest cannot be parsed successfully.
type: integer
resource:
description: Resource is the resource type of the resource
description: Resource is the resource type of the resource.
type: string
uid:
description: |-
Expand Down
54 changes: 48 additions & 6 deletions config/crd/bases/placement.kubernetes-fleet.io_clusterresourcebindings.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -430,10 +430,15 @@ spec:
properties:
allowCoOwnership:
description: |-
AllowCoOwnership defines whether to apply the resource if it already exists in the target cluster and is not
solely owned by fleet (i.e., metadata.ownerReferences contains only fleet custom resources).
If true, apply the resource and add fleet as a co-owner.
If false, leave the resource unchanged and fail the apply.
AllowCoOwnership controls whether co-ownership between Fleet and other agents are allowed
on a Fleet-managed resource. If set to false, Fleet will refuse to apply manifests to
a resource that has been owned by one or more non-Fleet agents.
Note that Fleet does not support the case where one resource is being placed multiple
times by different CRPs on the same member cluster. An apply error will be returned if
Fleet finds that a resource has been owned by another placement attempt by Fleet, even
with the AllowCoOwnership setting set to true.
type: boolean
comparisonOption:
default: PartialComparison
Expand Down Expand Up @@ -533,9 +538,22 @@ spec:
are on the member clusters.
If configuration differences are found on a resource, Fleet will consider this as an apply
error, which might block rollout depending on the specified rollout strategy.
Use ComparisonOption setting to control how the difference is calculated.
ClientSideApply and ServerSideApply apply strategies only work when Fleet can assume
ownership of a resource (e.g., the resource is created by Fleet, or Fleet has taken over
the resource). See the comments on the WhenToTakeOver field for more information.
ReportDiff apply strategy, however, will function regardless of Fleet's ownership
status. One may set up a CRP with the ReportDiff strategy and the Never takeover option,
and this will turn Fleet into a detection tool that reports only configuration differences
but do not touch any resources on the member cluster side.
For a comparison between the different strategies and usage examples, refer to the
Fleet documentation.
enum:
Expand Down Expand Up @@ -618,8 +636,10 @@ spec:
* IfNoDiff: with this action, Fleet will apply the hub cluster manifests to the member
clusters if (and only if) pre-existing resources look the same as the hub cluster manifests.
This is a safer option as pre-existing resources that are inconsistent with the hub cluster
manifests will not be overwritten; in fact, Fleet will ignore them until the inconsistencies
manifests will not be overwritten; Fleet will ignore them until the inconsistencies
are resolved properly: any change you make to the hub cluster manifests would not be
applied, and if you delete the manifests or even the ClusterResourcePlacement itself
from the hub cluster, these pre-existing resources would not be taken away.
Expand Down Expand Up @@ -647,9 +667,29 @@ spec:
If appropriate, you may also delete the object from the member cluster; Fleet will recreate
it using the hub cluster manifest.
* Never: with this action, Fleet will not apply a hub cluster manifest to the member
clusters if there is a corresponding pre-existing resource. However, if a manifest
has never been applied yet; or it has a corresponding resource which Fleet has assumed
ownership, apply op will still be executed.
This is the safest option; one will have to remove the pre-existing resources (so that
Fleet can re-create them) or switch to a different
WhenToTakeOver option before Fleet starts processing the corresponding hub cluster
manifests.
If you prefer Fleet stop processing all manifests, use this option along with the
ReportDiff apply strategy type. This setup would instruct Fleet to touch nothing
on the member cluster side but still report configuration differences between the
hub cluster and member clusters. Fleet will not give up ownership
that it has already assumed though.
enum:
- Always
- IfNoDiff
- Never
type: string
type: object
clusterDecision:
Expand Down Expand Up @@ -933,6 +973,9 @@ spec:
description: |-
TargetClusterObservedGeneration is the generation of the resource on the target cluster
that contains the configuration differences.
This might be nil if the resource has not been created yet on the target cluster.
format: int64
type: integer
version:
Expand All @@ -943,7 +986,6 @@ spec:
- kind
- name
- observationTime
- targetClusterObservedGeneration
- version
type: object
maxItems: 100
Expand Down
Loading

0 comments on commit 9a3cd78

Please sign in to comment.