We'll provision multiple Kubernetes namespaces, namespace per team. For each namespace:
- There will be a associated GCP project for cloud infrastructure
- K8s service account (KSA) linked to Google Service Account (GSA) on GCP side using Workload Identity
- To illustrate this example, a pod communicating to Google Storage Bucket in GCP project
-
Create projects team-a and team-b dedicated for two teams. Give permissions to Config Connector to manage resources in these projects. You can aso uncomment part of the script to create groups and give them permissions to projects:
bash ../provision-projects.sh
-
Create K8s namespace dedicated to team-a resources:
kubectl apply -f team-a/resources-admin/k8s-namespace.yaml
-
Configure permissions for this team-a-user to edit standard resources in this namespace and also give them cnrm-manager role to edit Config Connector resoruces, such as buckets.
kubectl apply -f team-a/resources-admin/k8s-namespace-permissions.yaml
-
Repeat previous two steps for team-b-user:
kubectl apply -f team-b/resources-admin/k8s-namespace.yaml kubectl apply -f team-b/resources-admin/k8s-namespace-permissions.yaml
-
Verify that team-a-user can create pods or sqlinstances in team-a namespace, but cannot do this in team-b or default namespace:
$ kubectl auth can-i create pods --namespace team-a --as=team-a-user yes $ kubectl auth can-i create sqlinstances --namespace team-a --as=team-a-user yes $ kubectl auth can-i create sqlinstances --namespace team-b --as=team-a-user no $ kubectl auth can-i create sqlinstances --as=team-a-user no
-
Try to modify team-a namespace to point it to a different project. You will get an error:
error: namespaces "team-a" could not be patched: namespaces "team-a" is forbidden: User "team-a-user" cannot patch resource "namespaces" in API group "" in the namespace "team-a"
-
Create
team-a
resources, impersonating team member:kubectl apply -f team-a/resources-team --as=team-a-user
-
Verify that service account credentials are propagating automatically. Run a pod with
google/cloud-sdk
image:kubectl run -it \ --generator=run-pod/v1 \ --image google/cloud-sdk \ --serviceaccount ksa-bucket-team-a\ --namespace team-a \ team-a-ksa-test --as=team-a-user
-
Once on the pod, run the following to test your permissions:
gcloud auth list # google service account should be listed # create file echo some text > f1.txt # copy to bucket, shoudl succeed gsutil cp f1.txt gs://alexbu-kcc-multiteam-team-a-bucket # list files in bucket, should succeed gsutil ls gs://alexbu-kcc-multiteam-team-a-bucket
-
Replicate the same configuration for team-b:
kubectl apply -f team-b/resources-team --as=team-b-user