From 664a51fab2e26c61b99ee69692abf88c3d15c36b Mon Sep 17 00:00:00 2001 From: rashmi_kh Date: Tue, 10 Sep 2024 00:37:10 +0530 Subject: [PATCH 1/6] changes to derice minimum service account Signed-off-by: rashmi_kh --- docs/drafts/derive-serviceaccount.md | 358 +++++++++++++++++++++++++++ 1 file changed, 358 insertions(+) create mode 100644 docs/drafts/derive-serviceaccount.md diff --git a/docs/drafts/derive-serviceaccount.md b/docs/drafts/derive-serviceaccount.md new file mode 100644 index 000000000..9623c12e9 --- /dev/null +++ b/docs/drafts/derive-serviceaccount.md @@ -0,0 +1,358 @@ +# Derive minimal ServiceAccount required for ClusterExtension Installation and Management + +OLM v1 security stance (secure by default) + +Adhering to OLM v1's "Secure by Default" tenet, OLM v1 does not have the permissions necessary to install content. This follows the least privilege principle and reduces the chance of a [confused deputy attack](https://en.wikipedia.org/wiki/Confused_deputy_problem). Instead, a ServiceAccount must be provided by users to install and manage content. + +Explain installing a CE requires a Service Account + +OLMv1 does not provide cluster admin privileges by default for installing cluster extensions. It depends on the cluster extension developer to specify the exact permissions required for the management of any specific bundle. A ServiceAccount needs to be explicitly specified for installing and upgrading operators else will face errors when deploying your cluster extension. + +The ServiceAccount is specified in the ClusterExtension manifest as follows: + +```yaml +apiVersion: olm.operatorframework.io/v1alpha1 +kind: ClusterExtension +metadata: + name: argocd +spec: + source: + sourceType: Catalog + catalog: + packageName: argocd-operator + version: 0.6.0 + install: + namespace: argocd + serviceAccount: + name: argocd-installer +``` + +The cluster extension installer will need RBAC in order to be able to assign the controller the RBAC it requires. +In order to derive the minimal RBAC for the installer service account, you must specify the following permissions: +* ClusterRole with all the roles specified in the bundle ClusterServiceVersion. +* ClusterExtension finalizer +* Allow ClusterExtenstion to set blockOwnerDeletion ownerReferences +* create the controller deployment +* create the ClusterResourceDefnitions +* create the other manifest objects +* create the necessary Cluster/Roles for the controller to be able to perform its job. +* get, list, watch, update, patch, delete the specific resources that get created +* update finalizers on the ClusterExtension to be able to set blockOwnerDeletion and ownerReferences + + +The following ClusterRole rules are needed: +# Allow ClusterExtension to set blockOwnerDeletion ownerReferences +- apiGroups: [olm.operatorframework.io] + resources: [clusterextensions/finalizers] + verbs: [update] + resourceNames: [] +# Manage CRDs +- apiGroups: [apiextensions.k8s.io] + resources: [customresourcedefinitions] + verbs: [create, list, watch] +- apiGroups: [apiextensions.k8s.io] + resources: [customresourcedefinitions] + verbs: [get, update, patch, delete] + resourceNames: [, ..., ] +# Manage ClusterRoles +- apiGroups: [rbac.authorization.k8s.io] + resources: [clusterroles] + verbs: [create, list, watch] +- apiGroups: [rbac.authorization.k8s.io] + resources: [clusterroles] + verbs: [get, update, patch, delete] + resourceNames: [, ..., , , ..., ] +# Manage ClusterRoleBindings +- apiGroups: [rbac.authorization.k8s.io] + resources: [clusterrolebindings] + verbs: [create, list, watch] +- apiGroups: [rbac.authorization.k8s.io] + resources: [clusterrolebindings] + verbs: [get, update, patch, delete] + resourceNames: [, ..., , , ..., ] +* Rules defined in the CSV under `.spec.install.clusterPermissions` and `.spec.install.permissions` +* Rules to manage any other cluster-scoped resources shipped with the bundle +* Rules to manage any other namespace-scoped resources +* Rules to manage the deployment defined in the ClusterServiceVersion +* Rules to manage the service account used for the deployment +- apiGroups: [apps] + resources: [deployments] + verbs: [create, list, watch] + +- apiGroups: [apps] + resources: [deployments] + verbs: [get, update, patch, delete] + resourceNames: [argocd-operator-controller-manager] + +- apiGroups: [""] + resources: [serviceaccounts] + verbs: [create, list, watch] + +- apiGroups: [""] + resources: [serviceaccounts] + verbs: [get, update, patch, delete] + resourceNames: [argocd-operator-controller-manager] +``` + +Below is an example of the argocd installer with the necessary RBAC to deploy the ArgoCD ClusterExtension: + +```yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: argocd-installer-clusterrole +rules: +# Allow ClusterExtension to set blockOwnerDeletion ownerReferences +- apiGroups: [olm.operatorframework.io] + resources: [clusterextensions/finalizers] + verbs: [update] + resourceNames: [argocd] +# Manage ArgoCD CRDs +- apiGroups: [apiextensions.k8s.io] + resources: [customresourcedefinitions] + verbs: [create] +- apiGroups: [apiextensions.k8s.io] + resources: [customresourcedefinitions] + verbs: [get, list, watch, update, patch, delete] + resourceNames: + - appprojects.argoproj.io + - argocds.argoproj.io + - applications.argoproj.io + - argocdexports.argoproj.io + - applicationsets.argoproj.io +# Manage ArgoCD ClusterRoles and ClusterRoleBindings +- apiGroups: [rbac.authorization.k8s.io] + resources: [clusterroles] + verbs: [create] +- apiGroups: [rbac.authorization.k8s.io] + resources: [clusterroles] + verbs: [get, list, watch, update, patch, delete] + resourceNames: + - argocd-operator.v0-1dhiybrldl1gyksid1dk2dqjsc72psdybc7iyvse5gpx + - argocd-operator-metrics-reader + - argocd-operator.v0-22gmilmgp91wu25is5i2ec598hni8owq3l71bbkl7iz3 +- apiGroups: [rbac.authorization.k8s.io] + resources: [clusterrolebindings] + verbs: [create] +- apiGroups: [rbac.authorization.k8s.io] + resources: [clusterrolebindings] + verbs: [get, list, watch, update, patch, delete] + resourceNames: + - argocd-operator.v0-1dhiybrldl1gyksid1dk2dqjsc72psdybc7iyvse5gpx + - argocd-operator.v0-22gmilmgp91wu25is5i2ec598hni8owq3l71bbkl7iz3 +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: argocd-installer-rbac-binding +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: argocd-installer-rbac-clusterrole +subjects: +- kind: ServiceAccount + name: argocd-installer + namespace: argocd +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: argocd-installer-rbac-clusterrole +rules: +# ArgoCD's operator requires the following permissions, which means the +# installer also needs them in order to create ArgoCD's RBAC objects. +- apiGroups: [""] + resources: [configmaps] + verbs: ['*'] +- apiGroups: [""] + resources: [endpoints] + verbs: ['*'] +- apiGroups: [""] + resources: [events] + verbs: ['*'] +- apiGroups: [""] + resources: [namespaces] + verbs: ['*'] +- apiGroups: [""] + resources: [persistentvolumeclaims] + verbs: ['*'] +- apiGroups: [""] + resources: [pods] + verbs: ['*', get] +- apiGroups: [""] + resources: [pods/log] + verbs: [get] +- apiGroups: [""] + resources: [secrets] + verbs: ['*'] +- apiGroups: [""] + resources: [serviceaccounts] + verbs: ['*'] +- apiGroups: [""] + resources: [services] + verbs: ['*'] +- apiGroups: [""] + resources: [services/finalizers] + verbs: ['*'] +- apiGroups: [apps] + resources: [daemonsets] + verbs: ['*'] +- apiGroups: [apps] + resources: [deployments] + verbs: ['*'] +- apiGroups: [apps] + resources: [deployments/finalizers] + resourceNames: [argocd-operator] + verbs: [update] +- apiGroups: [apps] + resources: [replicasets] + verbs: ['*'] +- apiGroups: [apps] + resources: [statefulsets] + verbs: ['*'] +- apiGroups: [apps.openshift.io] + resources: [deploymentconfigs] + verbs: ['*'] +- apiGroups: [argoproj.io] + resources: [applications] + verbs: ['*'] +- apiGroups: [argoproj.io] + resources: [appprojects] + verbs: ['*'] +- apiGroups: [argoproj.io] + resources: [argocdexports] + verbs: ['*'] +- apiGroups: [argoproj.io] + resources: [argocdexports/finalizers] + verbs: ['*'] +- apiGroups: [argoproj.io] + resources: [argocdexports/status] + verbs: ['*'] +- apiGroups: [argoproj.io] + resources: [argocds] + verbs: ['*'] +- apiGroups: [argoproj.io] + resources: [argocds/finalizers] + verbs: ['*'] +- apiGroups: [argoproj.io] + resources: [argocds/status] + verbs: ['*'] +- apiGroups: [authentication.k8s.io] + resources: [tokenreviews] + verbs: [create] +- apiGroups: [authorization.k8s.io] + resources: [subjectaccessreviews] + verbs: [create] +- apiGroups: [autoscaling] + resources: [horizontalpodautoscalers] + verbs: ['*'] +- apiGroups: [batch] + resources: [cronjobs] + verbs: ['*'] +- apiGroups: [batch] + resources: [jobs] + verbs: ['*'] +- apiGroups: [config.openshift.io] + resources: [clusterversions] + verbs: [get, list, watch] +- apiGroups: [monitoring.coreos.com] + resources: [prometheuses] + verbs: ['*'] +- apiGroups: [monitoring.coreos.com] + resources: [servicemonitors] + verbs: ['*'] +- apiGroups: [networking.k8s.io] + resources: [ingresses] + verbs: ['*'] +- apiGroups: [oauth.openshift.io] + resources: [oauthclients] + verbs: [create, delete, get, list, patch, update, watch] +- apiGroups: [rbac.authorization.k8s.io] + resources: ['*'] + verbs: ['*'] +- apiGroups: [rbac.authorization.k8s.io] + resources: [clusterrolebindings] + verbs: ['*'] +- apiGroups: [rbac.authorization.k8s.io] + resources: [clusterroles] + verbs: ['*'] +- apiGroups: [route.openshift.io] + resources: [routes] + verbs: ['*'] +- apiGroups: [route.openshift.io] + resources: [routes/custom-host] + verbs: ['*'] +- apiGroups: [template.openshift.io] + resources: [templateconfigs] + verbs: ['*'] +- apiGroups: [template.openshift.io] + resources: [templateinstances] + verbs: ['*'] +- apiGroups: [template.openshift.io] + resources: [templates] + verbs: ['*'] +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + name: argocd-installer-role + namespace: argocd +rules: +- apiGroups: [""] + resources: [serviceaccounts] + verbs: [create] +- apiGroups: [""] + resources: [serviceaccounts] + verbs: [get, list, watch, update, patch, delete] + resourceNames: [argocd-operator-controller-manager] +- apiGroups: [""] + resources: [configmaps] + verbs: [create] +- apiGroups: [""] + resources: [configmaps] + verbs: [get, list, watch, update, patch, delete] + resourceNames: [argocd-operator-manager-config] +- apiGroups: [""] + resources: [services] + verbs: [create] +- apiGroups: [""] + resources: [services] + verbs: [get, list, watch, update, patch, delete] + resourceNames: [argocd-operator-controller-manager-metrics-service] +- apiGroups: [apps] + resources: [deployments] + verbs: [create] +- apiGroups: [apps] + resources: [deployments] + verbs: [get, list, watch, update, patch, delete] + resourceNames: [argocd-operator-controller-manager] +``` + +# Creation of ClusterRoleBinding using the cluster-admin ClusterRole in non-production environments + +```yaml +# Create ClusterRole +kubectl apply -f - < Date: Tue, 10 Sep 2024 00:47:55 +0530 Subject: [PATCH 2/6] remove headers Signed-off-by: rashmi_kh --- docs/drafts/derive-serviceaccount.md | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/docs/drafts/derive-serviceaccount.md b/docs/drafts/derive-serviceaccount.md index 9623c12e9..e349ec131 100644 --- a/docs/drafts/derive-serviceaccount.md +++ b/docs/drafts/derive-serviceaccount.md @@ -41,12 +41,15 @@ In order to derive the minimal RBAC for the installer service account, you must The following ClusterRole rules are needed: -# Allow ClusterExtension to set blockOwnerDeletion ownerReferences +* Allow ClusterExtension to set blockOwnerDeletion ownerReferences + - apiGroups: [olm.operatorframework.io] resources: [clusterextensions/finalizers] verbs: [update] resourceNames: [] -# Manage CRDs + +* Manage CRDs + - apiGroups: [apiextensions.k8s.io] resources: [customresourcedefinitions] verbs: [create, list, watch] @@ -54,7 +57,9 @@ The following ClusterRole rules are needed: resources: [customresourcedefinitions] verbs: [get, update, patch, delete] resourceNames: [, ..., ] -# Manage ClusterRoles + +* Manage ClusterRoles + - apiGroups: [rbac.authorization.k8s.io] resources: [clusterroles] verbs: [create, list, watch] @@ -62,7 +67,9 @@ The following ClusterRole rules are needed: resources: [clusterroles] verbs: [get, update, patch, delete] resourceNames: [, ..., , , ..., ] -# Manage ClusterRoleBindings + +* Manage ClusterRoleBindings + - apiGroups: [rbac.authorization.k8s.io] resources: [clusterrolebindings] verbs: [create, list, watch] @@ -70,11 +77,13 @@ The following ClusterRole rules are needed: resources: [clusterrolebindings] verbs: [get, update, patch, delete] resourceNames: [, ..., , , ..., ] + * Rules defined in the CSV under `.spec.install.clusterPermissions` and `.spec.install.permissions` * Rules to manage any other cluster-scoped resources shipped with the bundle * Rules to manage any other namespace-scoped resources * Rules to manage the deployment defined in the ClusterServiceVersion * Rules to manage the service account used for the deployment + - apiGroups: [apps] resources: [deployments] verbs: [create, list, watch] @@ -92,7 +101,7 @@ The following ClusterRole rules are needed: resources: [serviceaccounts] verbs: [get, update, patch, delete] resourceNames: [argocd-operator-controller-manager] -``` + Below is an example of the argocd installer with the necessary RBAC to deploy the ArgoCD ClusterExtension: From 24d93f109cea5cd46f7a8e692f263835ff44f733 Mon Sep 17 00:00:00 2001 From: rashmi_kh Date: Tue, 10 Sep 2024 15:33:44 +0530 Subject: [PATCH 3/6] add details about registry+v1 support --- docs/drafts/derive-serviceaccount.md | 77 +++++++++++++++++----------- 1 file changed, 46 insertions(+), 31 deletions(-) diff --git a/docs/drafts/derive-serviceaccount.md b/docs/drafts/derive-serviceaccount.md index e349ec131..cd4088a4a 100644 --- a/docs/drafts/derive-serviceaccount.md +++ b/docs/drafts/derive-serviceaccount.md @@ -1,14 +1,8 @@ # Derive minimal ServiceAccount required for ClusterExtension Installation and Management -OLM v1 security stance (secure by default) +OLMv1 does not provide cluster admin privileges by default for installing cluster extensions.This means that the installation process will require a service account with sufficient privileges to install the bundle. It depends on the cluster extension developer to specify the exact permissions required for the management of any specific bundle. A Service Account needs to be explicitly specified for installing and upgrading operators else will face errors when deploying your cluster extension. -Adhering to OLM v1's "Secure by Default" tenet, OLM v1 does not have the permissions necessary to install content. This follows the least privilege principle and reduces the chance of a [confused deputy attack](https://en.wikipedia.org/wiki/Confused_deputy_problem). Instead, a ServiceAccount must be provided by users to install and manage content. - -Explain installing a CE requires a Service Account - -OLMv1 does not provide cluster admin privileges by default for installing cluster extensions. It depends on the cluster extension developer to specify the exact permissions required for the management of any specific bundle. A ServiceAccount needs to be explicitly specified for installing and upgrading operators else will face errors when deploying your cluster extension. - -The ServiceAccount is specified in the ClusterExtension manifest as follows: +The Service Account is specified in the ClusterExtension manifest as shown below: ```yaml apiVersion: olm.operatorframework.io/v1alpha1 @@ -27,20 +21,31 @@ spec: name: argocd-installer ``` -The cluster extension installer will need RBAC in order to be able to assign the controller the RBAC it requires. -In order to derive the minimal RBAC for the installer service account, you must specify the following permissions: -* ClusterRole with all the roles specified in the bundle ClusterServiceVersion. -* ClusterExtension finalizer -* Allow ClusterExtenstion to set blockOwnerDeletion ownerReferences -* create the controller deployment -* create the ClusterResourceDefnitions -* create the other manifest objects -* create the necessary Cluster/Roles for the controller to be able to perform its job. -* get, list, watch, update, patch, delete the specific resources that get created -* update finalizers on the ClusterExtension to be able to set blockOwnerDeletion and ownerReferences +The initial stable version (v1.0.0) only supports FBC catalogs containing registry+v1 bundles. OLMv1 will not support all OLMv0 content. OLMv1 will only support bundles that meet the following criteria: +* AllNamespaces install mode is enabled +* No dependencies on other packages or GVKs +* No webhooks +* Does not make use of the OperatorConditions API + +### Required RBAC +The cluster extension installer should have the prerequisite permissions in order to be able to assign the controller the RBAC it requires. In order to derive the minimal RBAC for the installer service account, you must specify the following permissions: + +* ClusterRole with all the roles specified in the bundle ClusterServiceVersion. This includes all the + rules defined in the CSV under `.spec.install.clusterPermissions` and `.spec.install.permissions` +* ClusterRole to create and manage CustomResourceDefinitions +* Update finalizers on the ClusterExtension to be able to set blockOwnerDeletion and ownerReferences +* Permissions to create the controller deployment, this corresponds to the rules to manage the + deployment defined in the ClusterServiceVersion +* Permissions to create the other manifest objects, rules to manage any other cluster-scoped resources + shipped with the bundle +* Rules to manage any other namespace-scoped resources +* Permissions to create the necessary roles and rolebindings for the controller to be able to perform its job +* Get, list, watch, update, patch, delete the specific resources that get created + + +### Below are snippets of the ClusterRole rule definitions: -The following ClusterRole rules are needed: * Allow ClusterExtension to set blockOwnerDeletion ownerReferences - apiGroups: [olm.operatorframework.io] @@ -48,7 +53,7 @@ The following ClusterRole rules are needed: verbs: [update] resourceNames: [] -* Manage CRDs +* Manage Custom Resource Definitions - apiGroups: [apiextensions.k8s.io] resources: [customresourcedefinitions] @@ -58,7 +63,7 @@ The following ClusterRole rules are needed: verbs: [get, update, patch, delete] resourceNames: [, ..., ] -* Manage ClusterRoles +* Manage Cluster Roles - apiGroups: [rbac.authorization.k8s.io] resources: [clusterroles] @@ -68,7 +73,7 @@ The following ClusterRole rules are needed: verbs: [get, update, patch, delete] resourceNames: [, ..., , , ..., ] -* Manage ClusterRoleBindings +* Manage Cluster Role Bindings - apiGroups: [rbac.authorization.k8s.io] resources: [clusterrolebindings] @@ -78,11 +83,7 @@ The following ClusterRole rules are needed: verbs: [get, update, patch, delete] resourceNames: [, ..., , , ..., ] -* Rules defined in the CSV under `.spec.install.clusterPermissions` and `.spec.install.permissions` -* Rules to manage any other cluster-scoped resources shipped with the bundle -* Rules to manage any other namespace-scoped resources -* Rules to manage the deployment defined in the ClusterServiceVersion -* Rules to manage the service account used for the deployment +* Manage deployments - apiGroups: [apps] resources: [deployments] @@ -93,6 +94,9 @@ The following ClusterRole rules are needed: verbs: [get, update, patch, delete] resourceNames: [argocd-operator-controller-manager] + +* Manage service accounts used for the deployment + - apiGroups: [""] resources: [serviceaccounts] verbs: [create, list, watch] @@ -105,7 +109,7 @@ The following ClusterRole rules are needed: Below is an example of the argocd installer with the necessary RBAC to deploy the ArgoCD ClusterExtension: -```yaml +???+ note apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: @@ -337,7 +341,18 @@ rules: resourceNames: [argocd-operator-controller-manager] ``` -# Creation of ClusterRoleBinding using the cluster-admin ClusterRole in non-production environments +### Manual process for minimal RBAC creation + +There are no production tools available to help you understand the precise RBAC required for your cluster extensions. However, if you want to figure this manually you can try the below: + +* Create all the intial rbac and then iterate over the ClusterExtention failures, examining conditions and updating the RBAC to include the generated cluster role names (name will be in the failure condition). +Install the ClusterExtension, read the failure condition, update installer RBAC and iterate until you are out of errors +* You can get the bundle image, unpacking the same and inspect the manifests to figure out the required permissions. +* The `oc` cli-tool creates cluster roles with a hash in their name, query the newly created ClusterRole names, then reduce the installer RBAC scope to just the ClusterRoles needed (inc. the generated ones). You can achieve this by allowing the installer to get, list, watch and update any cluster roles. + + + +### Creation of ClusterRoleBinding using the cluster-admin ClusterRole in non-production environments ```yaml # Create ClusterRole @@ -356,7 +371,7 @@ subjects: namespace: my-cluster-extension-namespace EOF -``` +```sh kubectl create clusterrolebinding my-cluster-extension-installer-role-binding \ --clusterrole=cluster-admin \ --serviceaccount=my-cluster-extension-namespace:my-cluster-installer-service-account From fdf7e9d5551964b712eb0f085d5bdf23ccddf1e0 Mon Sep 17 00:00:00 2001 From: rashmi_kh Date: Tue, 10 Sep 2024 15:37:06 +0530 Subject: [PATCH 4/6] render yml correctly Signed-off-by: rashmi_kh --- docs/drafts/derive-serviceaccount.md | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/docs/drafts/derive-serviceaccount.md b/docs/drafts/derive-serviceaccount.md index cd4088a4a..62cb1305d 100644 --- a/docs/drafts/derive-serviceaccount.md +++ b/docs/drafts/derive-serviceaccount.md @@ -48,13 +48,16 @@ The cluster extension installer should have the prerequisite permissions in orde * Allow ClusterExtension to set blockOwnerDeletion ownerReferences +```yaml - apiGroups: [olm.operatorframework.io] resources: [clusterextensions/finalizers] verbs: [update] resourceNames: [] +``` * Manage Custom Resource Definitions +```yaml - apiGroups: [apiextensions.k8s.io] resources: [customresourcedefinitions] verbs: [create, list, watch] @@ -62,9 +65,11 @@ The cluster extension installer should have the prerequisite permissions in orde resources: [customresourcedefinitions] verbs: [get, update, patch, delete] resourceNames: [, ..., ] +``` * Manage Cluster Roles +```yaml - apiGroups: [rbac.authorization.k8s.io] resources: [clusterroles] verbs: [create, list, watch] @@ -72,9 +77,11 @@ The cluster extension installer should have the prerequisite permissions in orde resources: [clusterroles] verbs: [get, update, patch, delete] resourceNames: [, ..., , , ..., ] +``` * Manage Cluster Role Bindings +```yaml - apiGroups: [rbac.authorization.k8s.io] resources: [clusterrolebindings] verbs: [create, list, watch] @@ -82,9 +89,11 @@ The cluster extension installer should have the prerequisite permissions in orde resources: [clusterrolebindings] verbs: [get, update, patch, delete] resourceNames: [, ..., , , ..., ] +``` * Manage deployments +```yaml - apiGroups: [apps] resources: [deployments] verbs: [create, list, watch] @@ -93,10 +102,11 @@ The cluster extension installer should have the prerequisite permissions in orde resources: [deployments] verbs: [get, update, patch, delete] resourceNames: [argocd-operator-controller-manager] - +``` * Manage service accounts used for the deployment - + +```yaml - apiGroups: [""] resources: [serviceaccounts] verbs: [create, list, watch] @@ -105,11 +115,12 @@ The cluster extension installer should have the prerequisite permissions in orde resources: [serviceaccounts] verbs: [get, update, patch, delete] resourceNames: [argocd-operator-controller-manager] - +``` Below is an example of the argocd installer with the necessary RBAC to deploy the ArgoCD ClusterExtension: ???+ note +```yaml apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: From 14bdc6a0590de0edf7af3c694cf3eb06a043182d Mon Sep 17 00:00:00 2001 From: Rashmi Khanna Date: Tue, 8 Oct 2024 23:05:24 +0530 Subject: [PATCH 5/6] Update cluster_extension_install_test.go --- test/e2e/cluster_extension_install_test.go | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/test/e2e/cluster_extension_install_test.go b/test/e2e/cluster_extension_install_test.go index ed514a70b..4cebcc57c 100644 --- a/test/e2e/cluster_extension_install_test.go +++ b/test/e2e/cluster_extension_install_test.go @@ -200,10 +200,10 @@ func testInit(t *testing.T) (*ocv1alpha1.ClusterExtension, *catalogd.ClusterCata } func testCleanup(t *testing.T, cat *catalogd.ClusterCatalog, clusterExtension *ocv1alpha1.ClusterExtension, sa *corev1.ServiceAccount) { - require.NoError(t, c.Delete(context.Background(), cat)) + require.NoError(t, c.Delete(context.Background(), cat, client.PropagationPolicy("DeletePropagationForeground"))) require.Eventually(t, func() bool { err := c.Get(context.Background(), types.NamespacedName{Name: cat.Name}, &catalogd.ClusterCatalog{}) - return errors.IsNotFound(err) + return err != nil && errors.IsNotFound(err) }, pollDuration, pollInterval) require.NoError(t, c.Delete(context.Background(), clusterExtension)) require.Eventually(t, func() bool { From bec7d35c839a2fbc185269cad8b01497d3f2fe51 Mon Sep 17 00:00:00 2001 From: Rashmi Khanna Date: Thu, 10 Oct 2024 23:26:56 +0530 Subject: [PATCH 6/6] Delete docs/drafts directory --- docs/drafts/derive-serviceaccount.md | 393 --------------------------- 1 file changed, 393 deletions(-) delete mode 100644 docs/drafts/derive-serviceaccount.md diff --git a/docs/drafts/derive-serviceaccount.md b/docs/drafts/derive-serviceaccount.md deleted file mode 100644 index 62cb1305d..000000000 --- a/docs/drafts/derive-serviceaccount.md +++ /dev/null @@ -1,393 +0,0 @@ -# Derive minimal ServiceAccount required for ClusterExtension Installation and Management - -OLMv1 does not provide cluster admin privileges by default for installing cluster extensions.This means that the installation process will require a service account with sufficient privileges to install the bundle. It depends on the cluster extension developer to specify the exact permissions required for the management of any specific bundle. A Service Account needs to be explicitly specified for installing and upgrading operators else will face errors when deploying your cluster extension. - -The Service Account is specified in the ClusterExtension manifest as shown below: - -```yaml -apiVersion: olm.operatorframework.io/v1alpha1 -kind: ClusterExtension -metadata: - name: argocd -spec: - source: - sourceType: Catalog - catalog: - packageName: argocd-operator - version: 0.6.0 - install: - namespace: argocd - serviceAccount: - name: argocd-installer -``` - -The initial stable version (v1.0.0) only supports FBC catalogs containing registry+v1 bundles. OLMv1 will not support all OLMv0 content. OLMv1 will only support bundles that meet the following criteria: -* AllNamespaces install mode is enabled -* No dependencies on other packages or GVKs -* No webhooks -* Does not make use of the OperatorConditions API - -### Required RBAC - -The cluster extension installer should have the prerequisite permissions in order to be able to assign the controller the RBAC it requires. In order to derive the minimal RBAC for the installer service account, you must specify the following permissions: - -* ClusterRole with all the roles specified in the bundle ClusterServiceVersion. This includes all the - rules defined in the CSV under `.spec.install.clusterPermissions` and `.spec.install.permissions` -* ClusterRole to create and manage CustomResourceDefinitions -* Update finalizers on the ClusterExtension to be able to set blockOwnerDeletion and ownerReferences -* Permissions to create the controller deployment, this corresponds to the rules to manage the - deployment defined in the ClusterServiceVersion -* Permissions to create the other manifest objects, rules to manage any other cluster-scoped resources - shipped with the bundle -* Rules to manage any other namespace-scoped resources -* Permissions to create the necessary roles and rolebindings for the controller to be able to perform its job -* Get, list, watch, update, patch, delete the specific resources that get created - - -### Below are snippets of the ClusterRole rule definitions: - -* Allow ClusterExtension to set blockOwnerDeletion ownerReferences - -```yaml -- apiGroups: [olm.operatorframework.io] - resources: [clusterextensions/finalizers] - verbs: [update] - resourceNames: [] -``` - -* Manage Custom Resource Definitions - -```yaml -- apiGroups: [apiextensions.k8s.io] - resources: [customresourcedefinitions] - verbs: [create, list, watch] -- apiGroups: [apiextensions.k8s.io] - resources: [customresourcedefinitions] - verbs: [get, update, patch, delete] - resourceNames: [, ..., ] -``` - -* Manage Cluster Roles - -```yaml -- apiGroups: [rbac.authorization.k8s.io] - resources: [clusterroles] - verbs: [create, list, watch] -- apiGroups: [rbac.authorization.k8s.io] - resources: [clusterroles] - verbs: [get, update, patch, delete] - resourceNames: [, ..., , , ..., ] -``` - -* Manage Cluster Role Bindings - -```yaml -- apiGroups: [rbac.authorization.k8s.io] - resources: [clusterrolebindings] - verbs: [create, list, watch] -- apiGroups: [rbac.authorization.k8s.io] - resources: [clusterrolebindings] - verbs: [get, update, patch, delete] - resourceNames: [, ..., , , ..., ] -``` - -* Manage deployments - -```yaml -- apiGroups: [apps] - resources: [deployments] - verbs: [create, list, watch] - -- apiGroups: [apps] - resources: [deployments] - verbs: [get, update, patch, delete] - resourceNames: [argocd-operator-controller-manager] -``` - -* Manage service accounts used for the deployment - -```yaml -- apiGroups: [""] - resources: [serviceaccounts] - verbs: [create, list, watch] - -- apiGroups: [""] - resources: [serviceaccounts] - verbs: [get, update, patch, delete] - resourceNames: [argocd-operator-controller-manager] -``` - -Below is an example of the argocd installer with the necessary RBAC to deploy the ArgoCD ClusterExtension: - -???+ note -```yaml -apiVersion: rbac.authorization.k8s.io/v1 -kind: ClusterRole -metadata: - name: argocd-installer-clusterrole -rules: -# Allow ClusterExtension to set blockOwnerDeletion ownerReferences -- apiGroups: [olm.operatorframework.io] - resources: [clusterextensions/finalizers] - verbs: [update] - resourceNames: [argocd] -# Manage ArgoCD CRDs -- apiGroups: [apiextensions.k8s.io] - resources: [customresourcedefinitions] - verbs: [create] -- apiGroups: [apiextensions.k8s.io] - resources: [customresourcedefinitions] - verbs: [get, list, watch, update, patch, delete] - resourceNames: - - appprojects.argoproj.io - - argocds.argoproj.io - - applications.argoproj.io - - argocdexports.argoproj.io - - applicationsets.argoproj.io -# Manage ArgoCD ClusterRoles and ClusterRoleBindings -- apiGroups: [rbac.authorization.k8s.io] - resources: [clusterroles] - verbs: [create] -- apiGroups: [rbac.authorization.k8s.io] - resources: [clusterroles] - verbs: [get, list, watch, update, patch, delete] - resourceNames: - - argocd-operator.v0-1dhiybrldl1gyksid1dk2dqjsc72psdybc7iyvse5gpx - - argocd-operator-metrics-reader - - argocd-operator.v0-22gmilmgp91wu25is5i2ec598hni8owq3l71bbkl7iz3 -- apiGroups: [rbac.authorization.k8s.io] - resources: [clusterrolebindings] - verbs: [create] -- apiGroups: [rbac.authorization.k8s.io] - resources: [clusterrolebindings] - verbs: [get, list, watch, update, patch, delete] - resourceNames: - - argocd-operator.v0-1dhiybrldl1gyksid1dk2dqjsc72psdybc7iyvse5gpx - - argocd-operator.v0-22gmilmgp91wu25is5i2ec598hni8owq3l71bbkl7iz3 ---- -apiVersion: rbac.authorization.k8s.io/v1 -kind: ClusterRoleBinding -metadata: - name: argocd-installer-rbac-binding -roleRef: - apiGroup: rbac.authorization.k8s.io - kind: ClusterRole - name: argocd-installer-rbac-clusterrole -subjects: -- kind: ServiceAccount - name: argocd-installer - namespace: argocd ---- -apiVersion: rbac.authorization.k8s.io/v1 -kind: ClusterRole -metadata: - name: argocd-installer-rbac-clusterrole -rules: -# ArgoCD's operator requires the following permissions, which means the -# installer also needs them in order to create ArgoCD's RBAC objects. -- apiGroups: [""] - resources: [configmaps] - verbs: ['*'] -- apiGroups: [""] - resources: [endpoints] - verbs: ['*'] -- apiGroups: [""] - resources: [events] - verbs: ['*'] -- apiGroups: [""] - resources: [namespaces] - verbs: ['*'] -- apiGroups: [""] - resources: [persistentvolumeclaims] - verbs: ['*'] -- apiGroups: [""] - resources: [pods] - verbs: ['*', get] -- apiGroups: [""] - resources: [pods/log] - verbs: [get] -- apiGroups: [""] - resources: [secrets] - verbs: ['*'] -- apiGroups: [""] - resources: [serviceaccounts] - verbs: ['*'] -- apiGroups: [""] - resources: [services] - verbs: ['*'] -- apiGroups: [""] - resources: [services/finalizers] - verbs: ['*'] -- apiGroups: [apps] - resources: [daemonsets] - verbs: ['*'] -- apiGroups: [apps] - resources: [deployments] - verbs: ['*'] -- apiGroups: [apps] - resources: [deployments/finalizers] - resourceNames: [argocd-operator] - verbs: [update] -- apiGroups: [apps] - resources: [replicasets] - verbs: ['*'] -- apiGroups: [apps] - resources: [statefulsets] - verbs: ['*'] -- apiGroups: [apps.openshift.io] - resources: [deploymentconfigs] - verbs: ['*'] -- apiGroups: [argoproj.io] - resources: [applications] - verbs: ['*'] -- apiGroups: [argoproj.io] - resources: [appprojects] - verbs: ['*'] -- apiGroups: [argoproj.io] - resources: [argocdexports] - verbs: ['*'] -- apiGroups: [argoproj.io] - resources: [argocdexports/finalizers] - verbs: ['*'] -- apiGroups: [argoproj.io] - resources: [argocdexports/status] - verbs: ['*'] -- apiGroups: [argoproj.io] - resources: [argocds] - verbs: ['*'] -- apiGroups: [argoproj.io] - resources: [argocds/finalizers] - verbs: ['*'] -- apiGroups: [argoproj.io] - resources: [argocds/status] - verbs: ['*'] -- apiGroups: [authentication.k8s.io] - resources: [tokenreviews] - verbs: [create] -- apiGroups: [authorization.k8s.io] - resources: [subjectaccessreviews] - verbs: [create] -- apiGroups: [autoscaling] - resources: [horizontalpodautoscalers] - verbs: ['*'] -- apiGroups: [batch] - resources: [cronjobs] - verbs: ['*'] -- apiGroups: [batch] - resources: [jobs] - verbs: ['*'] -- apiGroups: [config.openshift.io] - resources: [clusterversions] - verbs: [get, list, watch] -- apiGroups: [monitoring.coreos.com] - resources: [prometheuses] - verbs: ['*'] -- apiGroups: [monitoring.coreos.com] - resources: [servicemonitors] - verbs: ['*'] -- apiGroups: [networking.k8s.io] - resources: [ingresses] - verbs: ['*'] -- apiGroups: [oauth.openshift.io] - resources: [oauthclients] - verbs: [create, delete, get, list, patch, update, watch] -- apiGroups: [rbac.authorization.k8s.io] - resources: ['*'] - verbs: ['*'] -- apiGroups: [rbac.authorization.k8s.io] - resources: [clusterrolebindings] - verbs: ['*'] -- apiGroups: [rbac.authorization.k8s.io] - resources: [clusterroles] - verbs: ['*'] -- apiGroups: [route.openshift.io] - resources: [routes] - verbs: ['*'] -- apiGroups: [route.openshift.io] - resources: [routes/custom-host] - verbs: ['*'] -- apiGroups: [template.openshift.io] - resources: [templateconfigs] - verbs: ['*'] -- apiGroups: [template.openshift.io] - resources: [templateinstances] - verbs: ['*'] -- apiGroups: [template.openshift.io] - resources: [templates] - verbs: ['*'] ---- -apiVersion: rbac.authorization.k8s.io/v1 -kind: Role -metadata: - name: argocd-installer-role - namespace: argocd -rules: -- apiGroups: [""] - resources: [serviceaccounts] - verbs: [create] -- apiGroups: [""] - resources: [serviceaccounts] - verbs: [get, list, watch, update, patch, delete] - resourceNames: [argocd-operator-controller-manager] -- apiGroups: [""] - resources: [configmaps] - verbs: [create] -- apiGroups: [""] - resources: [configmaps] - verbs: [get, list, watch, update, patch, delete] - resourceNames: [argocd-operator-manager-config] -- apiGroups: [""] - resources: [services] - verbs: [create] -- apiGroups: [""] - resources: [services] - verbs: [get, list, watch, update, patch, delete] - resourceNames: [argocd-operator-controller-manager-metrics-service] -- apiGroups: [apps] - resources: [deployments] - verbs: [create] -- apiGroups: [apps] - resources: [deployments] - verbs: [get, list, watch, update, patch, delete] - resourceNames: [argocd-operator-controller-manager] -``` - -### Manual process for minimal RBAC creation - -There are no production tools available to help you understand the precise RBAC required for your cluster extensions. However, if you want to figure this manually you can try the below: - -* Create all the intial rbac and then iterate over the ClusterExtention failures, examining conditions and updating the RBAC to include the generated cluster role names (name will be in the failure condition). -Install the ClusterExtension, read the failure condition, update installer RBAC and iterate until you are out of errors -* You can get the bundle image, unpacking the same and inspect the manifests to figure out the required permissions. -* The `oc` cli-tool creates cluster roles with a hash in their name, query the newly created ClusterRole names, then reduce the installer RBAC scope to just the ClusterRoles needed (inc. the generated ones). You can achieve this by allowing the installer to get, list, watch and update any cluster roles. - - - -### Creation of ClusterRoleBinding using the cluster-admin ClusterRole in non-production environments - -```yaml -# Create ClusterRole -kubectl apply -f - <