Access a node without ssh

[till]

Was wondering if anyone went down this route of accessing a node without ssh, but e.g. by starting a container that mounts all the necessary directories (like root, proc, etc.). I know bottlerocket does something similar with admin and control containers, but has anyone done something similar for Flatcar?

[pothos]

There is kubectl debug node/mynode -it --image=busybox which has the node rootfs mounted as /host.

If that's not enough you can also define your own priv. pod. In the past I did something like kubectl run -it --rm test --restart=Never --image=debian:stable --overrides='{"spec": {"hostNetwork": true, "privileged": true, "containers": [ { "name": "test1", "image": "debian:stable", "command": ["/bin/bash"], "stdin": true, "tty": true, "securityContext": { "privileged": true, "capabilities": { "add": ["NET_ADMIN"] } } } ] }} '

[pothos]

As long as you have the same pid namespace, you can do nsenter -a -t 1 in a container with the real root user and you'll have a shell on the host.

[till]

@pothos sorry, should have said, these nodes don't use k8s. It's currently a Swarm cluster. I can remotely deploy containers/services etc. so I was looking for a solution there.

[till]

OK, for whoever comes next:

1. I deployed a service (mode: global) so it gets deployed to each node. Had to make sure the docker.sock is mounted (:ro)
2. Then I exec into the task to start a shell on the node:

docker run --privileged --pid=host -it alpine:3.8 nsenter -t 1 -m -u -n -i bash

And now I can run commands:

node-001 / # systemctl status docker
● docker.service - Docker Application Container Engine
     Loaded: loaded (8;;file://node-001.xxx/run/systemd/system/docker.service^G/run/systemd/system/docker.servic>
     Active: active (running) since Mon 2022-03-21 20:54:11 UTC; 1 year 1 month ago
TriggeredBy: ● docker.socket
       Docs: 8;;http://docs.docker.com^Ghttp://docs.docker.com8;;^G
   Main PID: 2454221 (dockerd)
      Tasks: 204
     Memory: 685.9M
     CGroup: /system.slice/docker.service
             ├─2454221 /run/torcx/bin/dockerd --host=f
...