2nd fullnode seems slow & unresponsive

[PFC-developer]

so I have a small kubeadm cluster, and have managed to get a single RPC node up.
but when I changed the replicas to 2, the 2nd node came up, but only process's blocks very slowly

the chain.yaml is available here
and the log output is available here
it is quite a few blocks behind.

ghcr.io/strangelove-ventures/heighliner/gaia:v11.0.0
ghcr.io/strangelove-ventures/cosmos-operator:v0.16.0

 kubectl get all -n cosmos -o wide
NAME              READY   STATUS    RESTARTS   AGE   IP           NODE                       NOMINATED NODE   READINESS GATES
pod/cosmoshub-0   2/2     Running   0          46h   10.0.4.46    w-f-01.xxx.loan   <none>           <none>
pod/cosmoshub-1   1/2     Running   0          27m   10.0.2.146   w-h-01.xxx.loan   <none>           <none>

NAME                      TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                                         AGE   SELECTOR
service/cosmoshub-p2p-0   NodePort    10.97.153.212    <none>        26656:30953/TCP                                 47h   app.kubernetes.io/instance=cosmoshub-0
service/cosmoshub-p2p-1   ClusterIP   10.104.194.226   <none>        26656/TCP                                       80m   app.kubernetes.io/instance=cosmoshub-1
service/cosmoshub-rpc     ClusterIP   10.106.64.48     <none>        1317/TCP,8080/TCP,9090/TCP,26657/TCP,9091/TCP   47h   app.kubernetes.io/name=cosmoshub

[PFC-developer]

after an hour the thing appeared to reset itself and both pods are 'ready 2/2'

[DavidNix]

This seems like an environment specific issue. Also, the operator does not control the processing of blocks; that's the chain software. We are not able to offer environment-specific support. If you'd like us to offer support, I can get you in touch with our biz dev team to come up with a support contract.

It appears you have enough CPU/RAM and using some sort of SSD for the disk. We use Google's premium-rwo which is suitable for syncing blocks.

Slow syncing can also be caused by pod having slow or not enough peers.