Unable to connect to the server: EOF when i try to login to the cluster

[dborgi]

While I was working on my okd cluster with version 4.13, suddenly stopped working, first the web console, then when I did a get I got this error: You must be logged in to the server (Unauthorized), When I tried to log in I got this message

oc login https://api.cluster.domain.com:6443 -u kubeadmin -p password --loglevel=10

Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"configmaps "motd" is forbidden: User "system:anonymous" cannot get resource "configmaps" in API group "" in the namespace "openshift"","reason":"Forbidden","details":{"name":"motd","kind":"configmaps"},"code":403}
I0718 14:50:32.248284 203653 helpers.go:264] Connection error: Head https://oauth-openshift.apps.cluster.domain.com: EOF
Unable to connect to the server: EOF

oc version
Client Version: 4.13.0-0.okd-2023-06-24-145750
Kustomize Version: v4.5.7
error: You must be logged in to the server (Unauthorized)

kubectl version
WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short. Use --output=yaml|json to get the full version.
Client Version: version.Info{Major:"1", Minor:"26", GitVersion:"v1.26.1", GitCommit:"e4c9a6aea", GitTreeState:"clean", BuildDate:"2023-06-21T22:04:17Z", GoVersion:"go1.19.9", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v4.5.7
error: You must be logged in to the server (the server has asked for the client to provide credentials)

Please if someone can help me thanks

[vrutkovs]

Does kubeconfig generated by installer still work? See also https://access.redhat.com/solutions/4845381 for other kubeconfigs to ensure the service is still up

[dborgi]

How do I know if kubeconfig is working?

[vrutkovs]

Check oc get co to see cluster status. If its broken please collect a must-gather and link it here for inspection

[dan1el-k]

Original solution: https://access.redhat.com/solutions/5953441

1. ssh to master node
2. execute following steps

#switch to root
sudo -i
#load kubeconfig from master node
export KUBECONFIG=/etc/kubernetes/static-pod-resources/kube-apiserver-certs/secrets/node-kubeconfigs/lb-int.kubeconfig
#e.g. oc get csr -o name | xargs oc adm certificate approve 

[dan1el-k]

Usually this happens when you have issues on your kube-apiserver, potentially caused by issue on etc-d or networking (ovn-kubemaster pods).
Or you have unapproved certificates open
Would suggest to further check the status of:

oc get clusteroperators
oc get csr -o name

and continue debugging from there