Error from the csi-provisioner containter once CrossNamespaceVolumeDataSource:true is enabled

[rajeeshckr]

/kind bug

When I enabled CrossNamespaceVolumeDataSource feature flag for the csi-provisioner container this way, --set sidecars.provisioner.additionalArgs={--feature-gates=CrossNamespaceVolumeDataSource=true} , it enabled the feature. But started getting this error message.

ebs-csi-controller-6559c897dc-6rwn7 csi-provisioner W0918 01:09:53.994057 1 reflector.go:533] k8s.io/[email protected]/tools/cache/reflector.go:231: failed to list *v1beta1.ReferenceGrant: referencegrants.gateway.networking.k8s.io is forbidden: User "system:serviceaccount:kube-system:ebs-csi-controller-sa" cannot list resource "referencegrants" in API group "gateway.networking.k8s.io" at the cluster scope

Fix is to update this file with

- apiGroups: ["gateway.networking.k8s.io"]
    resources: ["referencegrants"]
    verbs: ["get", "list", "watch"]

As per document

Environment

• Kubernetes version (use kubectl version):
  WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short. Use --output=yaml|json to get the full version. Client Version: version.Info{Major:"1", Minor:"27", GitVersion:"v1.27.2", GitCommit:"7f6f68fdabc4df88cfea2dcf9a19b2b830f1e647", GitTreeState:"clean", BuildDate:"2023-05-17T14:20:07Z", GoVersion:"go1.20.4", Compiler:"gc", Platform:"darwin/amd64"} Kustomize Version: v5.0.1 Server Version: version.Info{Major:"1", Minor:"26", GitVersion:"v1.26.7", GitCommit:"84e1fc493a47446df2e155e70fca768d2653a398", GitTreeState:"clean", BuildDate:"2023-07-19T12:16:45Z", GoVersion:"go1.20.6", Compiler:"gc", Platform:"linux/amd64"}
• Driver version:
  chart_version = "v2.22.0"
  app_version = "v1.22.0"

[rajeeshckr]

I have the changes in my fork

[AndrewSirenko]

Hi @rajeeshckr, thanks for bringing this up (and having an example of the changes in your fork, very thoughtful).

Discussed this with the EBS internal team, and because CrossNamespaceVolumeDataSource is an alpha and optional feature, there was some pushback on adding this attacher cluster role rule to the default helm chart.

However, in #1757, I have added "additionalClusterRoleRules" params to each relevant sidecar that would let you easily add this fix in. Tested with:

sidecars:
  provisioner:
    ...
    # Grant additional permissions to external-provisioner
    additionalClusterRoleRules:
    - apiGroups: ["gateway.networking.k8s.io"]
      resources: ["referencegrants"]
      verbs: ["get", "list", "watch"]

Does this satisfy your needs?