Can't deleted CFCR because the kubelet drain script is failing

[lucaspinto]

What happened: I was not able to delete a KuBo deployment because the drain script failed since it could not reach the master node when it came to execute the kubectl cordon command (https://github.com/cloudfoundry-incubator/kubo-release/blob/master/jobs/kubelet/templates/bin/drain.erb#L21)

What you expected to happen: When I delete a KuBo deployment, I expect the worker nodes to ignore the absence of a master node because it can be that this one gets deleted before the worker. There used to be a way to avoid this issue introduced in this commit: 79b6618#diff-34eaab0d59f5a3b9bd22cd961a1176d9

How to reproduce it (as minimally and precisely as possible): deploy cfcr with a lot of workers then delete it. Kubelet drain will fail rapidely.

Anything else we need to know?:

Environment:

• Deployment Info:

cfcr-lucas  bosh-dns/1.11.0   bosh-vsphere-esxi-ubuntu-xenial-go_agent/315.41  70 cloud/default    -
            bpm/1.0.4                                                          23 runtime/default
            cfcr-etcd/1.11.1
            docker/35.2.1
            kubo/0.34.0

• Environment Info:

Name               name
UUID               bc4044b2-ab6c-4f5f-942f-1330dc2f1989
Version            268.7.0 (00000000)
Director Stemcell  ubuntu-xenial/250.29
CPI                vsphere_cpi
Features           compiled_package_cache: disabled
                   config_server: enabled
                   local_dns: enabled
                   power_dns: disabled
                   snapshots: disabled
User               admin

• Kubernetes version:

Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.0", GitCommit:"e8462b5b5dc2584fdcd18e6bcfe9f1e4d970a529", GitTreeState:"clean", BuildDate:"2019-06-19T16:40:16Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"}

• Cloud provider: vsphere