Issues with redpanda operator

[mdfaizsiddiqui]

Hi

I am trying to setup a redpanda cluster with redpanda-operator, but facing couple of problems -

1. There was an issue with redpanda-operator helm chart where the annotation part was not indented properly, I have raised an issue and a PR to fix the issue.
2. Now, after this I stuck with another issue where i see below error creating Redpanda Cluster object -

one or more objects failed to apply, reason: admission webhook "mcluster.kb.io" denied the request: v1alpha1.Cluster.Spec: v1alpha1.ClusterSpec.Configuration: v1alpha1.RedpandaConfig.KafkaAPI: readObjectStart: expect { or n, but found [, error found in #10 byte of ...|afkaApi":[{"port":90|..., bigger context ...|:[{"port":9644}],"developerMode":true,"kafkaApi":[{"port":9092}],"pandaproxyApi":[{"port":8082}],"rp|... (retried 2 times).

Sharing all the necessary information to reproduce the issue ---

1. I am on EKS 1.27
2. Installing CRDs and operator via ArgoCD

---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: redpanda-crds
  namespace: argocd
  finalizers:
    - resources-finalizer.argocd.argoproj.io
spec:
  project: default
  destination:
    namespace: astradot
    server: https://kubernetes.default.svc
  source:
    repoURL: https://github.com/redpanda-data/redpanda.git
    targetRevision: HEAD
    path: src/go/k8s/config/crd
  syncPolicy:
    syncOptions:
      - CreateNamespace=true
      - ServerSideApply=true
    automated:
      prune: true
      selfHeal: true
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: redpanda-operator
  namespace: argocd
  finalizers:
    - resources-finalizer.argocd.argoproj.io
spec:
  project: default
  destination:
    namespace: astradot
    server: https://kubernetes.default.svc
  source:
    path: src/go/k8s/helm-chart/charts/redpanda-operator
    repoURL: "https://github.com/mdfaizsiddiqui/redpanda.git"
    targetRevision: HEAD
    helm:
      releaseName: redpanda-operator
      values: |
        affinity:
          nodeAffinity:
            requiredDuringSchedulingIgnoredDuringExecution:
              nodeSelectorTerms:
                - matchExpressions:
                    - key: karpenter.sh/provisioner-name
                      operator: Exists
                    - key: kubernetes.io/arch
                      operator: In
                      values: ["amd64"]
  syncPolicy:
    syncOptions:
      - CreateNamespace=true
      - ServerSideApply=true
    automated:
      prune: true
      selfHeal: true

3. Creating Cluster & Console Object (this is where I am getting the above error) -

---
apiVersion: redpanda.vectorized.io/v1alpha1
kind: Cluster
metadata:
  name: redpanda
  namespace: astradot
  annotations:
    argocd.argoproj.io/sync-wave: "1201"
spec:
  image: "redpandadata/redpanda"
  version: "latest"
  replicas: 1
  resources:
    requests:
      cpu: 1
      memory: 1.2Gi
    limits:
      cpu: 1
      memory: 1.2Gi
  configuration:
    kafkaApi:
      - port: 9092
    rpcServer:
      port: 33145
    pandaproxyApi:
      - port: 8082
    schemaRegistry:
      port: 8081
    adminApi:
      - port: 9644
    developerMode: true
---
apiVersion: redpanda.vectorized.io/v1alpha1
kind: Console
metadata:
  name: console
  namespace: astradot
  annotations:
    argocd.argoproj.io/sync-wave: "1202"
spec:
  server:
    listenPort: 8080
  schema:
    enabled: true
  clusterRef:
    name: redpanda
    namespace: astradot
  deployment:
    image: vectorized/console:master-173596f
  connect:
    enabled: true
    clusters:
      - name: console-connector
        url: http://localhost:8083
        tls:
          enabled: true
          secretKeyRef:
            name: connector-tls
            namespace: default

Can I know why I am seeing the error, is there a compatibility issue with the new EKS version, is it not supported? or there some genuine issue with the setup?

[emaxerrno]

@joejulian or @alejandroEsc or @mattschumpert may know.

[mdfaizsiddiqui]

Hey @emaxerrno @joejulian @alejandroEsc, @mattschumpert Any update on this? Please let me know if you need any additional information, thanks!

[joejulian]

I'm not that familiar with ArgoCD. Does your CRD installation stanza use kustomize? https://github.com/redpanda-data/redpanda/tree/dev/src/go/k8s/config/crd is expected to be rendered with kustomize.

[mdfaizsiddiqui]

Yes, it is rendered with kustomize. ArgoCD auto detects the presence of kustomization.yaml and use that to generate the manifest. Just to confirm I see Kustomize related parameter in ArgoCD for rp operator crd -

[mdfaizsiddiqui]

This is the kustomize version it is using -

argocd@argocd-repo-server-79ddcd68cf-5k9nn:~$ kustomize version
v5.0.1